Recovery planning 101 – what to do when the backup generator fails

On February 7th and 8th, 2019, credit unions and other businesses throughout the West Michigan area faced back-to-back ice storms leaving many in the dark. Soon after activating their continuity and recovery plan due to the loss of utility power, credit union core data processor CU*Answers experienced a major challenge: the failure of the backup generator. Operating on UPS backup power, recovery teams were able to perform an emergency high-availability failover to systems on the secondary data center in Yankton, SD, 750 miles away from the ice storms in Michigan. At the same time, fast-acting teams were able to diagnose and restore the generator before power was completely lost, maintaining operations until a reserve generator could be located and delivered later that day. “Our first priority,” said Jim Lawrence, Vice President of Business Continuity, “was the preservation of data and availability of support services during the disruption. While some non-critical systems were suspended to conserve UPS battery supply, the core platform was brought online at the secondary data center so that the backlog of transactions could process, and member credit unions could resume operations.” Business continuity principles and objectives are embedded into the corporate culture at CU*Answers. Regular recovery testing and training exercises help to prepare staff for unexpected disruptions enabling a prompt and effective response. Information regarding the CU*Answers Business Continuity program including tips for credit unions to improve their level of preparedness can be found on the CU*Answers website.