[tt] Microsoft's data centers growing by the truckload

[Brian Atkins]

http://news.cnet.com/8301-10805_3-10020902-75.html?hhTest=1

Once upon a time, Microsoft used to fill its data centers one server at a time. 
Then it bought them by the rack. Now it's preparing to load up servers by the 
shipping container.

Starting with a Chicago-area facility due to open later this year, Microsoft 
will use an approach in which servers arrive at the data center in a sealed 
container, already networked together and ready to go. The container itself is 
then hooked up to power, networking, and air conditioning.

"The trucks back 'em in, rack 'em, and stack 'em," Chief Software Architect Ray 
Ozzie told CNET News. And the containers remain sealed, Ozzie said. Once a 
certain number of servers in the container have failed, it will be pulled out 
and sent back to the manufacturer and a new container loaded in.

It's just one way that Microsoft is trying to cope in a world where it adds 
roughly 10,000 servers a month.

"You contain your infrastructure but you also contain the heat that's generated 
from the servers," Arne Josefsberg, Microsoft's general manager of 
infrastructure, said in an interview this week. "We are working incredibly hard 
to improve the energy efficiency of our data centers."

Only a couple of years ago, Microsoft was adding capacity one server at a time, 
adding individual servers to racks and taking a couple of hours to wire in each 
new server.

"That's way too expensive, way too slow," said Josefsberg.

Microsoft also used to lease much of its space, until it realized that data 
centers were going to be a very big part of its future as more and more software 
moved into the cloud. A couple of years back, though, it found itself running 
tight on capacity and bought two San Francisco Bay Area data centers in which it 
had been leasing space.

Over the past 18 months, though, Microsoft has been on a buying--and 
building--spree. The company has opened a data center in Quincy, Wash., and will 
open the Chicago facility, as well as another in San Antonio, Texas, later this 
year. A facility is due to open in Dublin, Ireland next year.

Microsoft is close to announcing yet another data center, Josefsberg said. The 
software maker also has signed a memorandum of understanding to build a data 
center in Russia.

No more off-the-shelf hardware
Gone are the days in which Microsoft settled for off-the-shelf hardware to fill 
its server farms. These days, Microsoft is looking for servers designed to its 
exact needs. It's not just that Microsoft doesn't want servers that have 
keyboard or USB ports--it wants motherboards that don't even have the added 
wiring necessary to support those things that it will never use. Such moves 
eliminate cost, space, and power consumption.

"We are not physically building our servers, but there is very deep engagement 
(with the computer makers)," Josefsberg said.

Even a 1 percent or 2 percent reduction in power consumption makes a big 
difference, Josefsberg said. As it is, Microsoft is trying to cram a whole lot 
of gear into a small space. While server racks at a Web hosting facility might 
have power densities of 70 watts to 100 watts per square foot, things are packed 
far more tightly in the containers, which might be consuming in the thousands of 
watts of power per square foot.

The container approach is easiest to implement on the ground floor of a 
facility. In Chicago, for example, it will use containers on the first floor and 
more traditional racks on the second level. But Josefsberg said that, though it 
poses some logistical challenges, the company is also considering using multiple 
levels of containers at other sites, including at a Dublin, Ireland data center 
due to open next year.

What the servers are serving up
So just what are all these data centers doing? Outsiders got a glimpse into this 
thanks to a slide included as part of a video that Microsoft put on its Web site 
touting its environmental efforts. The chart shows search accounting for the 
vast number of the servers--nearly 80,000 or so--with Hotmail and Messenger 
distant runners-up in terms of server usage.

Josefsberg said the figures were accurate, but out of date, reflecting where 
things were at a year or 18 months ago.

"Search was a very large portion of our demand in fiscal year 2008," he said. 
"Going into this year it is still a very large proportion. It is now not as 
dominant as it was last year."

Microsoft is seeing new demands, he said, such things like consumer video and 
photo services as well as its collection of hosted enterprise services under the 
Microsoft Online moniker.

Josefsberg said his goal is to keep capacity a certain number of months ahead of 
where Microsoft's utilization is running. To do that, he said, takes some 
serious planning. Business unit heads who used to have to just create a forecast 
for revenue and headcount, now need to be able to predict how much server 
capacity they will need, or at least give Josefsberg the data he needs to make 
such calculations.

He points to things like Microsoft's work with the Olympics as indicative of the 
kinds of demands his data centers will see in years to come.

"One of the big drivers for us that I see is the move to IP-based delivery of 
rich video," Josefsberg said.

And not all of his problems will be solved just because Microsoft can now get 
its servers by the containerful. Microsoft has sophisticated "heat maps" that 
plot the best locations for new data centers based on everything from government 
policy to water supply to power prices. But in other areas, such as networking 
technology, Microsoft is counting on the industry to make some quantum leaps.

"When you think about large-scale data centers there are a number of limitations 
in the technology," he said. "Some of the network protocols were designed years 
ago...Some are 30 years old."

-- 
Brian Atkins
Singularity Institute for Artificial Intelligence
http://www.singinst.org/