Paid Peering

Peering and interconnections are a mystery to most Internet users, who simply trust that the packets magically move from the web server to their desktop. A recent dispute over peering terms has left many Cogent customers experiencing network problems, and has drawn attention to the economics of peering, which we wrote about last May.

The dispute between Cogent and AOL has been widely discussed for several weeks on Internet mailing lists, including the North American Network Operators Group and several of Internet.com's ISP lists. Many Internet professionals noticed congestion issues once AOL discontinued its free peering arrangement with Cogent in mid-December.

Yesterday the dispute went public in a story in the Washington Post, with finger-pointing all around. Cogent says AOL is out of line in asking it to pay for peering. AOL says it never had a direct peering agreement with Cogent - its agreement was with PSINet, whose assets were subsequently bought out of bankruptcy by Cogent.

Peering is a critical cost containment tool for ISPs, but it also creates delicate business relationships in which traffic monitoring provides each party with an ongoing cost/benefit analysis. If it's not a win-win for both parties, the peering relationship is endangered.

AOL clearly believed its unpaid peering arrangement with Cogent was out of balance, with several reports suggesting the Cogent-to-AOL traffic was three times the volume of the AOL-to-Cogent traffic. Will it make sense for Cogent to pay for peering, as AOL wants? The outcome of this dispute bears watching, as it likely won't be the last time a peering disagreement clogs the pipes for customers.

Government Acquirers

Local governments are becoming increasingly active players in the telecom facilities game, and are paying prices for finished data center assets that exceed many marketplace notions of their value. The county commissioners of Sarasota County, Florida this week approved spending $3.5 million to purchase and equip a former Arthur Andersen data center.

The county will pay $1.75 million for the 14,108 square foot finished data center and equipment including 300 servers, and pump another $1.02 million into the facility to convert it to their use. The remainder of the costs reflect the assumption of an existing lease and employee salaries, according to local media reports.

The $1.75 million purchase price is about $124 a square foot. When you add in the additional retrofit cost, that number is closer to $196 a square foot - all of which gives the county the right to assume the lease at $280,000 a year (nearly $20 a square foot).

That struck the Sarasota County commissioners as a bargain. "The county's auditor, Ernst & Young, analyzed the deal and said the servers and other information technology equipment and the data center's specialized equipment, such as backup power systems, would bring $3.27 million if sold in a bankruptcy," according to the story in the Sarasota Herald Tribune.

This comes on the heels of last week's move by the Kansas City (Mo.) council to issue $44 million in bonds to fund the purchase and upgrade of an existing Ameritrade data center for use by Mastercard. The facility is 60,000 square feet, suggesting an eye-popping price for the completed facility of better than $700 a square foot. Even when you remove the incidentals from the deal, you're left with a large number.

In recent weeks, CarrierHotels.com and its affiliate Node Com have spent a lot of time researching data center deals completed in the past 18 months. Discussions of valuation and existing sales involve "apples vs. oranges" comparisons, as transactions vary widely in structure (fee-owned vs. leases, buildings with tenants versus empty sites) and often include a combination of cash and non-cash considerations.

But one thing that's clear from the recent government purchases is that end users are willing to pay for these assets, even in a buyers' market with plenty of supply.

While investors and corporate users are free to shop in several cities at once, government deals are driven by geography. If the goal is to create local jobs (as in Kansas City), the jobs need to be local to meet that goal. If a government wants to use the facility itself (as in Sarasota), it can't very well invest residents' tax dollars in another jurisdiction.

Silver Linings

Is colocation making a comeback? At least one consultancy thinks so, and a survey of service providers suggests that optimism is growing among industry insiders. An article Monday on Light Reading summarizes projections that colo may be poised to rebound in Europe.

The new research from Frost & Sullivan says the European colo market "can realistically achieve and maintain profitability, albeit at a vastly reduced pace than previous industry estimates implied" due to consolidation and cost-cutting efforts of surviving providers.

The study comes complete with a bar chart showing the familiar "hockey stick" upward growth curve that we saw on so many research reports during the boom years. Frost & Sullivan analyst Marina Martin predicts "steady increments in growth of around 30 per cent from the year 2005."

Regular readers know that I'm skeptical of these kind of projections. We report them, to be sure. But technology research firms were all too willing to fan the flames of irrational exuberance when the market was hot. Just like Wall Street stock analysts, they have to live with their track records.

The other piece of data that emerged this week was the November Service Provider Confidence Index from Sage Research, which was more intriguing and merits a closer look. The survey found that while inudstry professionals see business conditions deteriorating further, they are becoming more optimistic about the future. That hopefulness may be based on a trend found within the numbers. Sage's survey of 125 North American service providers found significant upticks in current conditions in two key categories - capital spending and current revenues.

"The continued upward trend shows a remarkable streak of optimism among service providers, despite the ever-worsening current conditions," says Chris Neal, a research director at Sage Research, Inc. "The fact that both the current conditions and expectations indices for capital spending and revenues are up sharply is a large part of the reason service provider see tomorrow's blue skies behind today's clouds."

Punditry is great, but experience is better. The bottom line seems to be that as the consolidation accelerates, the providers that have survived have no illusions about industry conditions, but feel better about the prospect that their endurance and tenacity will result in a long-term payoff.

Refugee Promotions

The battle for displaced hosting customers is heating up, as Telehouse America and Switch & Data this week announced "refugee promotions" that waive setup fees and offer a month or more of free colocation services.

With Cable & Wireless and Qwest each having announced major consolidations of their data center networks in recent weeks, competitors are customizing offers to attract new business. As long-time industry players who have avoided bankruptcy, Telehouse and Switch & Data are hoping to benefit from the turbulence.

Telehouse calls its promotion a "rescue package," and features a waiver of all standard installation fees, one month of free colocation space, and a guaranteed turnaround time of three days on a proposal.

“We have been receiving inquiries from concerned businesses, most of them small to mid-size hosting companies, seeking alternative colocation providers as a result of the data center closings,” said Hideki Akazawa, president and CEO of TELEHOUSE America. “We feel it is important to give these organizations short-term priority treatment so that they can quickly resolve their colocation problems in their time of need.”

Switch & Data is offering a "Colocation Migration Program" with free installation and two months of free colo for customers who sign a contract longer than 24 months.

"Our Colocation Migration Promotion assists customers facing impending data center closures to help ease the transition to Switch and Data services," said Mario Galvez, Vice President of Marketing at Switch and Data.

We'll no doubt see similar offers from other providers in coming weeks. Shop carefully, though. Today's migration promotion can turn into tomorrow's migration headache. It's worth noting that one of the providers with an active customer migration program has been Qwest.

Taxpayer Funding?

Looking for financing for a data center purchase and build-out? Just ask local taxpayers! Sound far-fetched? It really happened last week, when officials in Kansas City committed $44 million to attract MasterCard to establish a data center in the city.

On Nov. 28 the Kansas City Council unanimously approved a $44 million bond offering to help MasterCard locate a disaster recovery backup site in an area building. The funds would be used for the purchase and tenant improvements for a 60,000 square foot facility at 11530 N.W. Ambassador Drive near Kansas City International Airport owned by Ameritrade.

The new site would support MasterCard's new, $140 million disaster recovery center in St. Louis. MasterCard is currently mirroring settlement data from St. Louis to a site in Lake Success, N.Y., but the company has been searching for a new location closer to St. Louis.

Back in September, Ameritrade announced that it would sell the Kansas City data center to a "a major U.S. financial services firm" and lease back space from the new owner to continue its operations. It appears that MasterCard, understanding the local appetite for high-tech jobs, reached out to the Kansas City Council.

The resulting bond deal employs Chapter 100 incentives, which would return 50 percent of the new income and property taxes generated by the project in return for a 10-year subsidy. Platte County Economic Development Director Pete Fullerton said the center would generate $3 million in property taxes for the Park Hill School District and more than $1 million for the county annually by 2013.

"This is a facility that was going to get ramped down," Fullerton told the Platte County Sun. "All you have to do is look at your stock portfolio to see that. This caps off a deal that brings a stable, known quantity to a facility that could've ended up as empty as the one next to it."