[info] 8-node PS3 cluster

[Eugen Leitl]

(unfortunately both the GBit Ethernet NIC and the nVidia chips
is virtualized by the hypervisor)

http://www.wired.com/print/techbiz/it/news/2007/10/ps3_supercomputer

Astrophysicist Replaces Supercomputer with Eight PlayStation 3s

By Bryan Gardiner Email 10.17.07 | 12:00 AM

Gaurav Khanna’s eight PlayStation 3s aren’t running Heavenly Sword -- they’re
using Linux plus custom code to solve complex computations.

Photo: Courtesy of Gaurav Khanna

Suffering from its exorbitant price point and a dearth of titles, Sony's
PlayStation 3 isn't exactly the most popular gaming platform on the block.
But while the console flounders in the commercial space, the PS3 may be
finding a new calling in the realm of science and research.

Right now, a cluster of eight interlinked PS3s is busy solving a celestial
mystery involving gravitational waves and what happens when a super-massive
black hole, about a million times the mass of our own sun, swallows up a
star.

As the architect of this research, Dr. Gaurav Khanna is employing his
so-called "gravity grid" of PS3s to help measure these theoretical gravity
waves -- ripples in space-time that travel at the speed of light -- that
Einstein's Theory of Relativity predicted would emerge when such an event
takes place.

It turns out that the PS3 is ideal for doing precisely the kind of heavy
computational lifting Khanna requires for his project, and the fact that it's
a relatively open platform makes programming scientific applications
feasible.

"The interest in the PS3 really was for two main reasons," explains Khanna,
an assistant professor at the University of Massachusetts, Dartmouth who
specializes in computational astrophysics. "One of those is that Sony did
this remarkable thing of making the PS3 an open platform, so you can in fact
run Linux on it and it doesn't control what you do."

He also says that the console's Cell processor, co-developed by Sony, IBM and
Toshiba, can deliver massive amounts of power, comparable even to that of a
supercomputer -- if you know how to optimize code and have a few extra
consoles lying around that you can string together.

"The PS3/Linux combination offers a very attractive cost-performance solution
whether the PS3s are distributed (like Sony and Stanford's Folding at home
initiative) or clustered together (like Khanna's), says Sony's senior
development manager of research and development, Noam Rimon.

According to Rimon, the Cell processor was designed as a parallel processing
device, so he's not all that surprised the research community has embraced
it. "It has a general purpose processor, as well as eight additional
processing cores, each of which has two processing pipelines and can process
multiple numbers, all at the same time," Rimon says.

This is precisely what Khanna needed. Prior to obtaining his PS3s, Khanna
relied on grants from the National Science Foundation (NSF) to use various
supercomputing sites spread across the United States "Typically I'd use a
couple hundred processors -- going up to 500 -- to do these same types of
things."

However, each of those supercomputer runs cost Khanna as much as $5,000 in
grant money. Eight 60 GB PS3s would cost just $3,200, by contrast, but Khanna
figured he would have a hard time convincing the NSF to give him a grant to
buy game consoles, even if the overall price tag was lower. So after tweaking
his code this past summer so that it could take advantage of the Cell's
unique architecture, Khanna set about petitioning Sony for some help in the
form of free PS3s.

"Once I was able to get to the point that I had this kind of performance from
a single PS3, I think that's when Sony started paying attention," Khanna says
of his optimized code.

Khanna says that his gravity grid has been up and running for a little over a
month now and that, crudely speaking, his eight consoles are equal to about
200 of the supercomputing nodes he used to rely on.

"Basically, it's almost like a replacement," he says. "I don't have to use
that supercomputer anymore, which is a good thing."

"For the same amount of money -- well, I didn't pay for it, but even if you
look into the amount of funding that would go into buying something like
eight PS3s -- for the same amount of money I can do these runs indefinitely."

The point of the simulations Khanna and his team at UMass are running on the
cluster is to see if gravitational waves, which have been postulated for
almost 100 years but have never been observed, are strong enough that we
could actually observe them one day. Indeed, with NASA and other agencies
building some very big gravitational wave observatories with the sensitivity
to be able to detect these waves, Khanna's sees his work as complementary to
such endeavors.

Khanna expects to publish the results of his research in the next few months.
So while PS3 owners continue to wait for a fuller range of PS3 titles and low
prices, at least they'll have some reading material to pass the time.