[info] [Beowulf] Cheap SDR IB

[Eugen Leitl]

----- Forwarded message from Mark Hahn <hahn at mcmaster.ca> -----

From: Mark Hahn <hahn at mcmaster.ca>
Date: Wed, 30 Jan 2008 08:58:18 -0500 (EST)
To: "Jeffrey B. Layton" <laytonjb at charter.net>
Cc: Beowulf Mailing list <beowulf at beowulf.org>
Subject: Re: [Beowulf] Cheap SDR IB

>Just in case you've missed the announcements:
>http://www.clustermonkey.net//content/view/222/1/

I'm always happy about new levels pricing agression, but I'm
a bit puzzled about for what kind of workloads this will matter.

whenever I ask about IB bandwidth, people always point fingers 
at weather codes, which apparently are fond of doing the transpose
in multi-dimension FFT's using all-to-all.  while convenient, this
seems a bit silly, since transpose is O(N) communications, not O(N^2).

higher bandwidth/node also makes sense if you're configuring fairly
fat nodes (many cores, probably also lots of ram).  but if you do that,
you also amortize the networking, so a cheaper IB setup matters less.

perhaps there are some extremely file-IO intensive workloads that 
can sustain ~1 GB/s, but I'd expect them to require some hefty 
fileserving hardware, which would also hide the IB cost.

IB for gaming?  I have one ratio: 1e-1/3e-6.  that's human reaction
time versus IB latency.

also, I think it's a bit disingenous to use 10G Chelsio TOE to compare,
rather than 10G Myri which is cheaper and faster.  also:
http://www.chelsio.com/sandia_benchmark_tech.html

finally, how the heck do you make Gb as slow as 120 us?

-Mark "not actually anti-IB" Hahn.

PS: does anyone have first-hand experience with ConnectX performance?
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit 
http://www.beowulf.org/mailman/listinfo/beowulf

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE