PSTSWM Paragon Point-to-Point Communication Performance

Performance Studies using

PSTSWM

Intel Paragon SWAP Performance

(unordered swap of 128KB message using NX)

Date/Person:	February 2, 1998 / P. Worley
Platform:	Intel Paragon XP/S 150 MP at Oak Ridge National Laboratory:
	1024 MP nodes (3 50-MHz iPSC/860 processors per node)
Environment:	Paragon OSF/1 Release 1.0.4 Server 1.4 R1_4_5
	f77/Paragon Paragon Version R5.0.3
Communication Library:	NX
SWAP size:	16384 REAL*8 floating point values each direction
Message size:	Largest - 16384 REAL*8 floating point values
	Smallest - 16 REAL*8 floating point values
Processors:	0 and 1
Latency Definition:	(T₁₀₂₄-T₅₁₂)/512
Results:

unordered swap using nonblocking send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 53.04 105.35 32.6%

10 iter. 58.07 104.96 29.2%

1 iter. w/overlap 46.26 113.58 39.8%

10 iter. w/overlap 55.81 112.46 30.3%

unordered swap using nonblocking receive

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 67.62 82.99 32.8%

10 iter. 65.97 83.07 25.4%

1 iter. w/overlap 72.23 93.37 38.1%

10 iter. w/overlap 71.56 89.16 43.9%

unordered swap using nonblocking send and receive

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 70.87 82.85 42.0%

10 iter. 66.45 82.81 25.8%

1 iter. w/overlap 76.72 121.91 57.0%

10 iter. w/overlap 84.07 116.38 40.7%

unordered swap using ready send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 70.71 167.32 17.9%

10 iter. 68.33 165.44 23.0%

1 iter. w/overlap 73.00 92.68 41.3%

10 iter. w/overlap 71.03 89.28 43.4%

unordered swap using nonblocking ready send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 69.44 166.08 24.5%

10 iter. 70.77 164.12 21.5%

1 iter. w/overlap 78.07 133.72 55.6%

10 iter. w/overlap 87.09 116.12 38.4%

DISCUSSION

Unordered simple swap is not a safe protocol to use with the NX communication library, as it can cause deadlock if system communication buffer space is exhausted. Therefore, we do not include it in these tests.
Message-passing performance under OSF on the Paragon degrades significantly when attempting simultaneous send and receive on the same node (due to thrashing when accessing an address table?). This can be seen by comparing bandwidth estimates for these unordered results with bandwidth estimates for the ordered SWAP protocols. (Note that this is not as significant a problem when running the SUNMOS operating system from Sandia National Laboratory on the Paragon.)

Performance Studies using

PSTSWM

Intel Paragon SWAP Performance

(unordered swap of 128KB message using NX)

DISCUSSION

Patrick H. Worley / ( worleyph@ornl.gov) Last Modified Monday, 15-Jul-2002 10:29:11 EDT. 86652 accesses since 1/2/96.

Patrick H. Worley / ( worleyph@ornl.gov)
Last Modified Monday, 15-Jul-2002 10:29:11 EDT.
86652 accesses since 1/2/96.