[tt] CHE: Energy Drain by Computers Stifles Efforts at Cost Control

[Premise Checker]

Energy Drain by Computers Stifles Efforts at Cost Control
http://chronicle.com/free/v55/i18/18a00103.htm
The Chronicle of Higher Education, 9.1.9

'We don't have a meter on the data center,' says one college official
By JOSH KELLER
Menlo Park, Calif.

For decades, the major computers at the Stanford Linear Accelerator Center 
have multiplied almost without limit.

Row by row, racks of computer servers have expanded outward in a constant 
quest to provide computing power for the center's data-intensive 
experiments. The servers have taken over new wings of an office building 
at the site, which is operated by Stanford University. Many of those 
unfortunate enough to work nearby have been displaced, and their former 
offices house towering black machines.

Now the pipes that supply cold water to help keep the servers cool are 
running at full capacity. The building has trouble taking in the huge 
amounts of electricity that modern-day servers require. For each dollar 
spent on computers, the center must spend an equal amount of money to 
build the power and cooling systems to keep them running.

That cost "has been killing us," says Richard P. Mount, the center's head 
of scientific computing. The price of storing and processing data, in 
fact, is hurting every college and university in the country.

In response, some institutions are embracing greener technologies, as much 
to keep costs down as to help the environment. Stanford is moving toward 
building a new center that uses outside air instead of chilled water, and 
it hopes to save just over $3-million per year. "Arguably, this pays for 
itself," says Phil Reese, the university's faculty and research computing 
strategist. "There's not many arguments you can give that are that 
strong." And there are other steps, like consolidating servers and 
outsourcing services, that are less expensive than building a new facility 
and reduce data's budget-devouring appetite, computer experts told The 
Chronicle.

But compared with other industries, colleges and universities have been 
slow to understand the problem and to adopt energy-saving techniques, 
experts say. Colleges are decentralized, and they often lack a cohesive 
strategy to reduce energy use. Researchers, in fact, often resist plans 
for centralization because they want their own servers just down the hall. 
As a result, many institutions waste millions of dollars per year powering 
inefficient machines, outdated cooling systems, and improvised clusters of 
servers stored in lab closets and back rooms.

One survey, for instance, found that higher-education institutions are 
less likely than businesses, the K-12 sector, and the federal government 
to have a formal policy to encourage energy-efficient buying decisions and 
are less likely to hold information-technology departments responsible for 
their own energy costs. The survey was conducted in June by the computer 
vendor CDW-G.

"We typically in higher education have not had the financial incentives 
lined up," said Mark S. Askren, assistant vice chancellor for 
administrative computing services at the University of California at 
Irvine. "Folks in data-center organizations like mine who are consuming 
energy, we don't have a meter on the data center. We don't even know how 
much we're consuming."

At Stanford, leaders realized the depth of the problem when plans for 
every new major building included requests for major computing facilities 
inside of them, says Mr. Reese. The requests were symptomatic of a larger 
problem that plagues many institutions, Mr. Reese says: Data centers are 
spread out across the campus, making it more difficult to ensure that the 
computing facilities are energy efficient.

In response, the university is moving toward building a new, greener data 
center off-campus, on the site of the Stanford Linear Accelerator Center. 
The new data center would serve the university's research needs as well as 
take over half of the computing capacity of the linear accelerator itself, 
alleviating some of its infrastructure problems.

The new facility would be twice as energy efficient as the university's 
current model, Mr. Reese says. The building is designed to take advantage 
of Northern California's temperate climate, cooling the servers with 
circulated outside air instead of using chilled water, which is expensive 
to cool down. It would also expel the hot air given off by the servers, 
reducing the need for external cooling.

Mr. Reese says he expects the data center could cost upwards of 
$50-million, but he said the center will save the university $3.2-million 
per year in energy costs over 25 years.

Stanford's proposed improvements are dramatic, but many efforts to 
increase efficiency do not require such a large initial investment. In 
interviews, data-center experts recommend steps college administrators 
could take to begin reducing a data center's energy costs.

Conduct an Audit

Some colleges do not understand how much money they are spending powering 
their data centers, and that makes it difficult to solve the problem.

For many, the first step is to conduct an energy audit. Representatives 
from facilities departments can typically measure how much power is being 
consumed at each of the campus's major data-center clusters, says Dallas 
Thornton, a division director for cyberinfrastructure services at the San 
Diego Supercomputer Center at the University of California at San Diego. 
They can then add up the readings to determine the total data-center 
energy usage of the campus.

"More times than not, it will be a high number," Mr. Thornton says.

Private consultants and some equipment vendors also offer efficiency 
audits, though Mr. Thornton says an effective audit can often be done 
in-house. A couple of common measurements have been developed to evaluate 
the approximate energy efficiency of a data center, which often focus on 
the excess energy used by the facility that houses the machines.

PUE, for example, which stands for power usage effectiveness, is 
calculated by taking the ratio of the energy use of the total data-center 
facility and the energy use of just the IT systems themselves. (The Green 
Grid, a prominent industry group that promotes energy-efficient data 
centers, explains the details of measuring data center efficiency at 
thegreengrid.org.)

Colleges can often save significant amounts of energy by making small 
improvements to their cooling systems, which can account for 50 percent of 
a data center's energy usage, says Jesse Hanz, a regional director in 
Pittsburgh for APC, an equipment vendor that offers energy audits to 
colleges.

"There's no better place to start in the data center to become more energy 
efficient, because there's a lot of waste there," says Mr. Hanz.

Establish a Working Group

The roadblocks to establishing more efficient data centers are as much 
organizational as they are technical. Historically, IT departments have 
been responsible for selecting and paying for data-center equipment, but 
facilities departments have been responsible for footing the electricity 
bill. Unless the two departments coordinate their budgets and set energy 
efficiency as a common goal, reducing usage is difficult.

"The people buying the computer equipment don't have the incentive to 
spend a dollar to get a more efficient computer even if they might save $5 
or $10 on more-efficient energy use," says Jonathan Koomey, a consulting 
professor at Stanford.

Long-term planning for operational costs could more easily be ignored when 
data centers were smaller and the most important things were speed and 
access to the equipment. But now that the costs are so high, establishing 
incentives to encourage energy-efficient behavior on the part of IT 
departments and faculty is imperative, Mr. Koomey says. On some campuses, 
IT departments that can prove their purchases saved money on operational 
costs recoup some of that money in their budgets.

Don Carli, a research fellow at the Institute for Sustainable 
Communication, an advocacy group, says reducing energy use requires 
setting common goals among many parts of an organization, always difficult 
on college campuses. Mr. Carli recommended forming a committee that has 
the power to create incentives that seek to encourage more 
energy-efficient behavior.

"By its very nature, sustainability is an interdisciplinary concept," says 
Mr. Carli. "It's a systems-thinking concept, and basically institutions of 
higher learning are a collection of silos."

Consolidate

One particularly inefficient strategy for grouping servers is common at 
colleges. Instead of using a centralized data center, research groups 
install their own servers locally, often in back rooms and closets. The 
set-up helps gives researchers greater access to their machines, but it 
can take 50 percent more energy than the same amount of resources at a 
centralized data center. Researchers call them "server closets" or "closet 
clusters."

Amin M. Vahdat, a professor of computer science at the University of 
California at San Diego, says the strategy is particularly wasteful 
because researchers might only use the servers every so often, when they 
have a paper due or need to perform calculations.

"I have my cluster and I only need it twice a year--well, I'm not going to 
power it down," says Mr. Vahdat, who is researching how to build more 
energy-efficient networks. "I build my cluster, put it in an 
energy-efficient closet, and use it twice a year, and I'm not 
exaggerating."

Forcing employees to use centralized facilities is much more difficult at 
a university than in the business world, which makes it hard for colleges 
to control energy costs, says Gregory Ganger, director of the Parallel 
Data Laboratory at Carnegie Mellon University. Researchers do not trust 
that IT departments will be able to fit their individual needs, and they 
like to have direct access to their machines.

"At a company, the company can dictate the software you're going to use," 
Mr. Ganger says. "At a university, theoretically the central 
administration could dictate the software you're going to use, but in 
practice that's impossible. ... You're not going to dictate to them, 'Thou 
shalt use the following version of Linux.'"

More and more types of researchers are depending on large amounts of 
computing power to perform their research, however, making a centralized 
data center for research all the more important. Mr. Ganger says the 
practice of virtualization, which allows researchers to establish 
different operating environments while using the same data center, has 
matured enough to work in a wider variety of situations.

New infrastructure is expensive, of course, and the recession is forcing 
some institutions to forgo building their own new data centers. Instead, 
they are looking at ways campuses can share their facilities to achieve 
greater economies of scale.

At the University of California, administrators are discussing 
establishing two data centers that will serve some of the system's 10 
campuses in each half of the state, says Mr. Thornton, at the San Diego 
Supercomputer Center. Taking a regional approach, he says, could allow the 
university to build fewer specialized facilities to power and cool the 
servers.

"It's something that in tough budget times that people are a lot more 
likely to be interested in," Mr. Thornton says. "It's hard to go to bat to 
get new funding for a data center right now."

Outsource

Some of the most energy-efficient data centers are those run by technology 
companies like Google and Microsoft and the online retailer Amazon.com, 
whose profits depend on finding cost-effective ways to store and process 
data. In the long term, experts expect many colleges to export much of 
their operations to companies like these to save money and focus on what 
they know best.

Already, more than 1,000 colleges have signed up for e-mail service 
through Google or Microsoft, helping those colleges reduce, if only 
slightly, their need for on-campus data centers. Despite some concerns 
about student privacy, many colleges have reported that letting 
professionals take care of e-mail results in significant savings.

A model for outsourcing more significant parts of the data center--the 
computing needed for scientific research, for instance--is still being 
developed, says Thomas A. DeFanti, a professor of computer science at the 
University of Illinois at Chicago. Even if some universities did figure 
out how to outsource more of their computing resources, he says, some do 
not have access to networks that would support efficiently moving such 
large amounts of data.

"There isn't a price model that everybody believes in yet, and it does of 
course depend on networking that's not available in some places," Mr. 
DeFanti says.

Still, Mr. Koomey, the Stanford consulting professor, says the idea of 
using external resources "makes a ton of sense."

"Why not let the people who have a lot of users do all the work?" he says. 
"It puts you at risk in some other ways, but for the most part there 
really are economies of scale."