PSTSWM T3E-900 Point-to-Point Communication Performance

Performance Studies using

PSTSWM

SGI/Cray Research T3E-900 SWAP Performance

(ordered swap of 2MB message using SHMEM)

(performance measured per processor when all processors in node communicating)

Date/Person:	September 30, 1999 / P. Worley
Platform:	T3E-900 at National Energy Research Scientific Computing Center (mcurie.nersc.gov):
	532 450-MHz DEC Alpha EV5 RISC processors
Environment:	UNICOS/mk 2.0.3.41
	mpt 1.2.1.3
	Cray CF90 Version 3.1.0.3
Communication Library:	SHMEM
SWAP size:	262144 REAL*8 floating point values each direction
Message size:	Largest - 262144 REAL*8 floating point values
	Smallest - 256 REAL*8 floating point values
Processors:	0 and 2
	1 and 3
Latency Definition:	(T₁₀₂₄-T₅₁₂)/512
Results:

ordered swap using nonblocking send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 326.70 7.29 2.1%

10 iter. 330.79 7.26 0.7%

1 iter. w/overlap 326.77 6.83 1.7%

10 iter. w/overlap 330.89 6.55 1.2%

ordered swap using nonblocking receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 348.10 9.81 17.9%

10 iter. 351.31 9.81 18.2%

1 iter. w/overlap 347.90 9.13 16.2%

10 iter. w/overlap 350.88 8.89 18.3%

ordered swap using nonblocking send and receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 327.22 7.23 3.7%

10 iter. 330.91 7.31 1.1%

1 iter. w/overlap 325.81 6.62 1.5%

10 iter. w/overlap 330.52 6.43 1.2%

ordered swap using ready send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 348.45 10.47 19.3%

10 iter. 351.55 10.09 17.7%

1 iter. w/overlap 346.92 9.54 17.4%

10 iter. w/overlap 350.51 9.36 17.7%

DISCUSSION

In these tests, we examine SHMEM communication performance between two pairs of processors, looking for signs of performance degradation due to network contention. A real test for network contention would use realistic communication patterns and more than four processors. However, this test is used as a control study for the tests on the systems using shared memory nodes, in which contention and other degradation occurs almost immediately.