PSTSWM Paragon Point-to-Point Communication Performance

Performance Studies using

PSTSWM

Intel Paragon SWAP Performance

(unordered swap of 2MB message using NX)

Date/Person:	January 30, 1998 / P. Worley
Platform:	Intel Paragon XP/S 150 MP at Oak Ridge National Laboratory:
	1024 MP nodes (3 50-MHz iPSC/860 processors per node)
Environment:	Paragon OSF/1 Release 1.0.4 Server 1.4 R1_4_5
	f77/Paragon Paragon Version R5.0.3
Communication Library:	NX
SWAP size:	262144 REAL*8 floating point values each direction
Message size:	Largest - 262144 REAL*8 floating point values
	Smallest - 256 REAL*8 floating point values
Processors:	0 and 1
Latency Definition:	(T₁₀₂₄-T₅₁₂)/512
Results:

unordered swap using nonblocking send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 54.76 79.87 21.2%

10 iter. 57.14 83.24 22.0%

1 iter. w/overlap 58.53 87.06 22.4%

10 iter. w/overlap 58.21 85.43 23.2%

unordered swap using nonblocking receive

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 74.91 36.51 27.6%

10 iter. 73.32 40.48 25.6%

1 iter. w/overlap 74.16 46.88 7.9%

10 iter. w/overlap 73.42 46.19 1.4%

unordered swap using nonblocking send and receive

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 75.77 31.73 30.6%

10 iter. 69.30 35.51 24.4%

1 iter. w/overlap 89.91 96.53 34.4%

10 iter. w/overlap 80.85 91.18 37.5%

unordered swap using ready send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 74.28 140.60 7.0%

10 iter. 72.67 150.51 10.4%

1 iter. w/overlap 73.41 46.07 5.6%

10 iter. w/overlap 73.58 46.52 2.4%

unordered swap using nonblocking ready send

Data Statistics
bidirectional bandwidth estimated latency model error

(peak MByte/sec) (usec/msg) (max. rel. error)

1 iter. 73.51 145.15 7.4%

10 iter. 72.85 151.68 10.9%

1 iter. w/overlap 84.88 100.45 40.7%

10 iter. w/overlap 82.44 91.40 35.3%

DISCUSSION

Unordered simple swap is not a safe protocol to use with the NX communication library, as it can cause deadlock if system communication buffer space is exhausted. Therefore, we do not include it in these tests.
Message-passing performance under OSF on the Paragon degrades significantly when attempting simultaneous send and receive on the same node (due to thrashing when accessing an address table?). This can be seen by comparing bandwidth estimates for these unordered results with bandwidth estimates for the ordered SWAP protocols. (Note that this is not as significant a problem when running the SUNMOS operating system from Sandia National Laboratory on the Paragon.)

Performance Studies using

PSTSWM

Intel Paragon SWAP Performance

(unordered swap of 2MB message using NX)

DISCUSSION

Patrick H. Worley / ( worleyph@ornl.gov) Last Modified Monday, 15-Jul-2002 10:29:09 EDT. 86245 accesses since 1/2/96.

Patrick H. Worley / ( worleyph@ornl.gov)
Last Modified Monday, 15-Jul-2002 10:29:09 EDT.
86245 accesses since 1/2/96.