PSTSWM Origin2000 Point-to-Point Communication Performance

Performance Studies using

PSTSWM

SGI Origin2000 SWAP Performance

(ordered swap of 2MB message using SHMEM)

(performance measured per processor when all processors in node communicating)

Date/Person:	September 30, 1999 / P. Worley
Platform:	SGI Origin2000 at Los Alamos National Laboratory:
	128 250-MHz MIPS R10000 processors
Environment:	IRIX 6.5
	mpt_1.3.0.0
	MIPSpro Compilers: Version 7.2.1
Communication Library:	SHMEM
SWAP size:	262144 REAL*8 floating point values each direction
Message size:	Largest - 262144 REAL*8 floating point values
	Smallest - 256 REAL*8 floating point values
Processors:	0 and 2
	1 and 3
Latency Definition:	(T₁₀₂₄-T₅₁₂)/512
Results:

ordered swap using nonblocking send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 165.20 6.00 9.9%

10 iter. 505.52 6.17 20.0%

1 iter. w/overlap 170.43 6.59 14.4%

10 iter. w/overlap 537.54 5.34 22.3%

ordered swap using nonblocking receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 171.97 6.16 9.0%

10 iter. 509.60 6.25 19.4%

1 iter. w/overlap 172.20 6.81 11.7%

10 iter. w/overlap 562.43 3.78 23.8%

ordered swap using nonblocking send and receive

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 164.31 5.77 12.6%

10 iter. 493.40 6.25 19.2%

1 iter. w/overlap 169.32 6.19 8.3%

10 iter. w/overlap 495.51 6.25 18.9%

ordered swap using ready send

Data Statistics
unidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 171.08 4.67 12.2%

10 iter. 512.66 6.28 19.6%

1 iter. w/overlap 171.26 6.22 7.1%

10 iter. w/overlap 512.28 6.22 19.5%

DISCUSSION

An Origin2000 processing node consists of one or two processors. If two processors are in a node, they share the node's network interface hardware. In these tests, we examine SHMEM communication performance between processors in different nodes when both processors in one node are communicating with both processors in the other node. The performance is reported for a single processor pair (logical processors 0 and 2), representing the latency and bandwdith that would typically appear in a highly synchronous communication-intensive operation, such as a collective communication command.
The experiments using multiple iterations show the effect of the improved memory locality intrinsic to the iteration logic, giving significantly better performance for the smaller message sizes. This is unlikely to be indicative of performance in real applications.