PSTSWM T3E-900 Point-to-Point Communication Performance

Performance Studies using

PSTSWM

SGI/Cray Research T3E-900 SWAP Performance

(ordered swap of 8KB message using SHMEM)

(performance measured per processor when all processors in node communicating)

Date/Person:	September 30, 1999 / P. Worley
Platform:	T3E-900 at National Energy Research Scientific Computing Center (mcurie.nersc.gov):
	532 450-MHz DEC Alpha EV5 RISC processors
Environment:	UNICOS/mk 2.0.3.41
	mpt 1.2.1.3
	Cray CF90 Version 3.1.0.3
Communication Library:	SHMEM
SWAP size:	1024 REAL*8 floating point values each direction
Message size:	Largest - 1024 REAL*8 floating point values
	Smallest - 1 REAL*8 floating point value
Processors:	0 and 2
	1 and 3
Latency Definition:	(T₁₀₂₄-T₅₁₂)/512
Results:

ordered swap using nonblocking send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 180.57 6.36 15.3%

10 iter. 252.34 5.64 17.4%

1 iter. w/overlap 161.90 5.91 17.0%

10 iter. w/overlap 241.90 5.97 17.6%

ordered swap using nonblocking receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 196.73 4.40 43.4%

10 iter. 260.65 4.95 15.8%

1 iter. w/overlap 193.48 5.49 21.6%

10 iter. w/overlap 275.89 5.13 17.3%

ordered swap using nonblocking send and receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 173.63 7.42 19.6%

10 iter. 251.97 5.76 17.7%

1 iter. w/overlap 166.19 6.17 12.5%

10 iter. w/overlap 244.87 5.90 17.6%

ordered swap using ready send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 193.15 6.30 14.8%

10 iter. 242.25 5.41 16.0%

1 iter. w/overlap 178.40 5.86 12.8%

10 iter. w/overlap 227.31 5.54 23.3%

DISCUSSION

In these tests, we examine SHMEM communication performance between two pairs of processors, looking for signs of performance degradation due to network contention. A real test for network contention would use realistic communication patterns and more than four processors. However, this test is used as a control study for the tests on the systems using shared memory nodes, in which contention and other degradation occurs almost immediately.