'What If' Becomes Real
Jacksonville power outage provides real-world test for Peak 10
May 2, 2002 -- It is the hypothetical "what if"
moment that justifies the existence of data centers. You never
know when it may arrive. But it's the reason behind the installation
of huge generators, expensive UPS systems and banks of batteries.
What if the power goes out or there's a natural disaster? Will
my company's data be safe?
"What if" arrived Monday afternoon in Jacksonville,
Fla., when a system failure left 355,000 customers of the Jacksonville
Electric Authority without electricity. For Peak 10 Technology
Gateways, it was the moment when all the contingency planning
came off the drawing board and into the data center.
all worked as planned and as practiced. But that's not always
In the past 18 months, data centers' reliability claims have been
tested by a series of disasters, including a 6.8 magnitude earthquake
in Seattle in Feb. 2001, flooding from Tropical Storm Allison
in Houston last June, and the Sept. 11 terrorist attacks in New
York and Washington.
On balance, the industry has performed exceptionally. Several
Houston data centers were flooded, and one NYC carrier hotel experienced
sustained outages in September. But the vast majority of facilities
have stayed online, and that was the case this week in Jacksonville.
It started innocently enough. At 3:50 p.m., a lightning arrestor
- a small device designed to protect transmission lines from lightning
strikes - failed near from the Pickettville Substation on the
west side of Jacksonville. Within minutes, that single failure
had triggered a cascading series of equipment problems that overwhelmed
the JEA's entire grid.
By 4:15 pm, many residents and businesses in Jacksonville were
At Peak 10's Jacksonville facility at 4905 Belfort Road, monitoring
equipment detected the loss of commercial power and switched to
battery power. Within seconds, the building's 750 KVa Caterpillar
diesel generator came to life, keeping the web servers up and
Meanwhile, the power supply fluctuated across the city as JEA
engineers sought to get the system back online. The first outage
at Peak 10 lasted just four minutes, followed by a two-hour period
of normal operation, and then a second outage that lasted 20 minutes.
"Our facilities are set up to stay on the generator until
the power has been restored and has stabilized for 30 minutes,"
said John Proctor, a senior engineer at Peak 10. "All in
all, we ran off the generator for about an hour and a half. We
keep 2,900 gallons of fuel on site, which will allow the facility
to run for 48 to 72 hours, depending on the load."
In addition, Peak 10’s service agreements from local fuel vendors
guarantee priority refueling in case of long-term power outages.
Things were worse elsewhere, as some areas of Jacksonville were
without power for 12 hours. City residents had to boil water for
several days afterward due to water pressure problems related
to the blackout.
While other Jacksonville businesses lost hours of productivity
and sales, Peak 10 customers remained online. For data centers,
that's business as usual.
"The power is always on at Peak 10,” said executive vice president
and COO David Jones. “Once a week, we simulate a power failure
and run all facilities off of our diesel generators to ensure
that our power systems will continue to operate seamlessly.”
"We're in the business of keeping customer networks and systems
up and running all of the time - even during major commercial
power failures," said Mike Raines, general manager of Peak 10's
Jacksonville site. “This is exactly why customers use Peak 10.”
Some members of the local business community have apparently noticed.
have entered into negotiations with several new prospects since
the outage," noted Proctor.