PSTSWM SP3-375 Serial Performance

Performance Studies using

PSTSWM

Date/Person:	February 18, 2000 / P. Worley
Platform:	IBM SP3 at Oak Ridge National Laboratory (eagle.ccs.anl.gov):
	62 4-way Winterhawk II SMP nodes (375 MHz POWER3 with 8MB L2 cache)
Environment:	AIX 4.3.3; PSSP 3.1.1
Code Version:	6.7
Make Options:	MACH=sp COMM=mpi PRECISION=8 PERF=n WORKSPACE=20000000 MATH=essl
Compilation Options:	mpxlf_r -O3 -qarch=auto -qtune=auto -qcache=auto
Link Options:	-bmaxdata:0x70000000
Number of steps:	T42: 241 or 481
	T85: 49 or 97
	T170: 49 or 97
Number of processors per node:	1, 2, 3, or 4
Notes:	using ESSL library routines for Fourier transforms and BLAS (-lessl)

T42 MFlop rates

Per Processor Rate

Per Node Rate

**T42 MFlop rates**
Per Processor Rate	Per Node Rate

T85 MFlop rates

Per Processor Rate

Per Node Rate

**T85 MFlop rates**
Per Processor Rate	Per Node Rate

T170 MFlop rates

Per Processor Rate

Per Node Rate

**T170 MFlop rates**
Per Processor Rate	Per Node Rate

The performance discrepancy here indicates the cost of contention for memory (bandwidth) when multiple processors are running simultaneously. The total memory required per node goes up as the number of processors increases, but the per processor cache behavior and memory access patterns is constant.
This experiment indicates the difference in performance one might see in a space sharing context, sharing the node with other users, or in a scaled speed-up experiment, where the size of a parallel job is increased as the number of processors used increases.
Note that two different metrics are displayed - per processor computational rate and per node computational rate. The per node rate is the best measure of the performance of the architecture, but the per processor rate is a convenient way to observe performance degradation.
Summary: Both the per processor and per node performance degrades significantly as a function of the number of vertical levels, and this degradation is faster the more processors per node that are used. In particular, the per node performance when using 4 processor per node is almost identical to that of for 3 processors per node for T85 and more than 3 vertical levels and for all T170 cases. It is never significantly worse, however.