See the hardware requirements for GAMMA.
Click here for an application-level performance comparison based on OpenFOAM 1.4.
Otherwise, find below some latency-bandwidth numbers (taken at user level).
The testbed is a Gigabit Ethernet cluster of dual-CPU PCs with Linux 2.6 and the GAMMA driver. Each PC contains two Intel Xeon 2.8GHz processors on a Tyan Tiger i7501s motherboard, which carries an on-board Intel 82546EB Gigabit Ethernet controller with UTP plugs over copper.
The cluster is networked by a 28+4-way Extreme Network Summit 7i Gigabit Ethernet switch.
The following definitions hold:
Message delay, D(S), is half the round-trip time with a message of size S bytes, as measured by running a ``Ping-Pong'' MPI microbenchmark.
Latency is the message delay D(Smin) where Smin is the smallest message size allowed by the communication system (0 bytes with MPI).
End-to-end throughput, T(S), is the transfer rate of the
whole communication path, from the sender to the receiver:
T(S) = S/D(S).
NOTE: This definition implies that the throughput is measured using the ping-pong microbenchmark. Other techniques for throughput measurement are based on the transmission of long streams of messages from sender to receiver without any data flowing back; such techniques will measure a different thing, that we call transmission throughput (see below).
Asymptotic bandwidth, B, is the value of the following ``bandwidth'' function B(S) as the message size S > 0 approaches infinity: B(S) = (S - Smin)/(D(S) - D(Smin)). Since B(S) - T(S) approaches zero as S approaches infinity, B is commonly evaluated by measuring the end-to-end throughput T(S) with S very large.
Transmission throughput is the transfer rate perceived at the sender side, that is, the data rate at which an infinite stream of messages of fixed size can be pushed into the network without causing any data loss. This requires to run a different microbenchmark compared to ping-pong: Rather than exchanging a single message back and forth, the sender transmits a long stream of messages to the receiver and measures the time spent for the whole transmission.
Half-power point, is the message size H at which the throughput T(H) reaches half the asymptotic bandwith B. The value of H depend on which notion of throughput one has in mind. We refer here to the end-to-end throughput.
The communication performance on such a cluster is as follows
(best among three runs of a MPI ping-pong microbenchmark, each providing an
average over 50 trials):
|Protocol||NIC||Latency||Latency||Asymptotic bandwidth||Half-power point||Half-power point|
|(back to back)||(switch incl.)||(Jumbo frames, MTU 4120 bytes)||(back to back)||(switch incl.)|
|MPI/GAMMA||Intel PRO/1000||6.5 µs||11.3 µs||122.9 MByte/s||1960 byte||7500 byte|
Before using MPI/GAMMA, you must install and test the most recent version of GAMMA in your cluster.
You also need to download MPICH version 1.2.7 subversion ``p1''. Click here to download.
And here is the current release of MPI/GAMMA, dated 13 January 2009.
If you intend to run MPI/GAMMA jobs through a batch scheduler like Torque/PBS, you need an utility for job submission. This is the source of a patched version of Pete Wyckoff's "mpiexec" utility version 0.83, suitable for use with MPI/GAMMA.
Before installing, please read the README_MPIGAMMA file, containing useful instructions. Should you need additional information (e.g. you find the installation instructions unclear or incomplete), please contact Giuseppe Ciaccio, ciaccio AT disi.unige.it.
Bugs and other problems may be reported to Giuseppe Ciaccio, ciaccio AT disi.unige.it