Emergency Preparedness + Response
CURRENT AND EMERGING TRENDS IN UTILITY DISASTER PLANNING
From flash floods and tornadoes, to hurricanes, earthquakes and mud slides, the number of natural disasters reported globally has increased considerably over the past three decades, according to the World Meteorological Organization. There were nearly 7,500 natural disasters between 1980 and 2005, which claimed more than two million lives and caused more than $1.2 trillion in losses.¹
Disasters such as Hurricane Sandy, the Fukushima earthquake and Hurricane Katrina delivered horrendous destruction. In many cases it will take years, if not decades, to repair the damage; and even then, the areas hit the hardest may never be the same.
The loss of lives and the destruction of physical structures are the most obvious and emotionally devastating result of disasters. However, another growing and serious risk — especially for businesses and governments — is the loss of critical data. As data is vital to how utilities operate, serve customers and work with regulatory bodies, the ability to recover data after a disaster should be a critical concern.
For utilities, including those in regulated and competitive energy markets, the amount and types of data collected from the grid, customers and other sources have grown exponentially. This data, and its use in informing better decisions, has allowed utilities to move beyond simply gathering customer information, billing and payment records and energy usage information. It’s now used to uncover operational efficiencies, improve customer engagement and increase program enrollment. Notably, 2013 data analytics award winners Pacific Gas and Electric, Oncor, South Carolina Electric & Gas² and Sacramento Municipal Utility District³ are using data in innovative new ways.
While utilities, especially those in regulated markets, aren’t subject to the risk of going bankrupt, compared to businesses that operate in competitive markets, a statistic published by the National Archives & Records Administration4 shed light on the potential devastation of data loss. According to its report, 93 percent of companies that lost their data center for 10 days or more due to a disaster, filed for bankruptcy within one year. Fifty percent of businesses that found themselves without data management for this same time period filed for bankruptcy immediately. Devastation from data loss for all types of businesses, including utilities, is quantifiable and not misplaced scare tactic hype.
With data loss so closely linked to business failure, data-focused disaster planning is nothing new for utilities, even before analytics-based opportunities entered the scene.
Tim Almond, a vice president for TMG Consulting, an organization that has been advising utilities on critical information technology investments since 1992, recalls, “Twenty-five years ago, the biggest concern for one of my utility clients was a data center bombing. We used to cycle full images of data through off-site secure locations. I used to load a bunch of disks in the trunk of my car and deposit them in a bomb and fire-proof safe out of town. Our disaster recovery plan back then was an agreement with the vendor who parked an 18-wheeler with a mainframe and HVAC system at the disaster recovery site of our choice.”
utilities should do their homework and understand how each cloud vendor adopts, implements, and maintains security controls; and whether these measures are adequate for their utility.
Clearly the information age has allowed disaster recovery to evolve beyond disks and 18-wheelers, to more efficient and accessible technology-based options. But data loss due to a natural disaster remains a risk for utilities that are hesitant to move beyond on-premises hardware and software. A more dispersed storage option is offered through the cloud.
When people talk about the cloud, or cloud computing, some may envision a cumulus type. While this might be an effective visual for marketing, it leads to a misconception that important data is floating somewhere in an intangible, virtual universe. It is this ambiguity that makes utilities nervous, and no one can blame them.
The reality of cloud computing is more defined: It is a business model that enables on-demand network access to a shared pool of configurable computing resources (such as networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or interactions with service providers.
These resources reside in physical locations, but the security of cloud computing, in part, can be attributed to a plurality of storage locations. Cloud solutions usually include multiple components that are stored in separate buildings, in multiple dispersed geographic locations. There has yet to be a natural disaster that has leveled all of these physical locations simultaneously.
In an Electric Public Research Institute (EPRI) whitepaper, The Utility Industries Cautious Move to the Cloud: an In Depth Look at the Benefits and Risks,5 Nihat Guven, director of strategic alliances for Qualys, a cloud security provider, gave his own perspective on the security of the cloud. He explained that reputable cloud service providers take security very seriously and apply multiple layers of controls to protect their clients’ data.
“The data is not stored in its complete form in any one place,” he said. “The key or a piece of the key may be stored in one place, the encrypted data in another and so on. It would take a conspiracy of unrealistic proportions to make this data accessible to anyone except the end user, and a natural disaster of even bigger proportions to completely lose the data.”
Guven warned that utilities should do their homework and understand how each cloud vendor adopts, implements, and maintains security controls; and whether these measures are adequate for their utility.
Cost is driving moves to cloud data storage. According to the article, Moving Disaster Recovery to the Cloud: Is it Your Best Move?6, the yearly cost of traditional IT infrastructure, business continuity and disaster recovery tops the list of the drivers to cloud computing.
Central Lincoln PUD (People’s Utility District), a publicly owned electric utility on the Oregon coast, is moving its meter data management (MDM) to the cloud, a final initiative covered under a $9.9 million investment grant from the American Recovery and Reinvestment Act of 2009 to build a smart grid.
The solution will perform long-term data storage and management for the vast quantities of data delivered by smart metering systems. According to the utility, this dataset consists primarily of usage data and events.
“Our decision to move our MDM to the cloud was due, in part, to the need to prepare for a massive subduction zone earthquake and tsunami that is predicted to hit the Pacific Northwest,” explained Bruce Lovelin, Central Lincoln PUD’s chief engineer/systems engineering manager. “Because of our location, we ask ourselves how we can continue to serve customers if a disaster leveled our physical locations.”
Central Lincoln PUD is ahead of the game. Last September, a task force submitted a plan to the Oregon legislature to spend more than $200 million every biennium in long-term disaster planning, including requirements for utilities.7
“Most of Oregon’s buildings, roads, bridges and infrastructure were built at a time when it was believed the state was not subject to major earthquakes,” said Scott Ashford, director of the Cascadia Lifelines Program, a research initiative to help improve critical infrastructure performance during an anticipated major earthquake on the Cascadia subduction zone. “Because of that, we’re going to face serious levels of destruction. But with programs like this, and the commitment of our partners, there’s a great deal we can do to proactively prepare for this disaster, and to get our lifelines back up and running after the event.”
Cascadia Lifelines Program partners include: the Oregon Department of Transportation, Portland General Electric, NW Natural, the Bonneville Power Administration, Port of Portland, Portland Water Bureau, Eugene Water and Electric Board, and Tualatin Valley Water District.
In addition to ensuring that your data will be protected by strict security policies and practices, choosing a cloud solution for disaster recovery should include asking other important questions such as:
While the cloud can deliver tremendous benefits, understanding all of the options to mitigate potential calamities should be considered by utilities as they talk with cloud solution providers about disaster recovery options.
Because it isn’t a matter of if, but when a natural disaster will occur, and data is so important to a utility’s ability to conduct business, data recovery planning needs to be an ongoing focus for utility IT departments.
Ashford, who has studied the impact of subduction zone earthquakes in much of the Pacific Rim — including Japan’s major disaster in March, 2011, said, “When I studied areas that had been hard-hit by earthquakes in Chile, New Zealand and Japan, it became apparent that money spent to prepare for and minimize damage from the earthquake was hugely cost-effective. This impressed upon me that we do not have to just wait for the earthquake to happen. There’s a lot we can do to prepare for it right now that will make a difference.”
Because of the cloud’s geographic dispersion, self-service, digital accessibility and elastic resources can be rapidly re-apportioned as needed, which is part of the “preparation” referenced by Ashford. Therefore, it’s likely that a growing number of utilities will join Central Lincoln PUD and others in the cloud. Opportunities for managing Customer Information Systems (CIS), automated dispatch and mobile data, work management, Advanced Metering Infrastructure (AMI) and Meter Data Management systems through this business model may be the best option for data recovery after the next big of disaster.