The SGI/Cray Research T3E-900 is a distributed-memory parallel
architecture built around a high-performance 3D torus interconnect.
We looked at the protocol sensitivities of two different message-passing
libraries: MPI and an application-specific library
built using the SHMEM one-sided communication operations
get and put.
The MPI and SHMEM communication protocols are not directly
comparable, but some general conclusions can be drawn from the protocol
sensitivity experiments. The primary conclusion is that there are few
similarities between the MPI and SHMEM results:
Most of the optimal MPI protocols do not even exist in the SHMEM
implementation.
The MPI results show much more performance sensitivity than the SHMEM
results.
The MPI results show strong similarities in performance sensitivity
between sets of algorithms. Corresponding similarities do not hold for the
SHMEM data.
Overlap protocols are often optimal when using SHMEM, but are less
useful when using MPI.
Some of these differences are a result of there being many more
communication protocols available in the MPI experiments than in the SHMEM
experiments. However, the SHMEM results demonstrate significantly better
performance, so nothing is being gained (in the way of performance)
from the larger number of MPI communication protocols.
The only significant agreement between the MPI and SHMEM results is that
that Experiments B and C show less performance variation than Experiment A.
There is also some similarity in how the performance variation varies -
proprotional to the number of processors (1D) and inversely proprotional to
the problem granularity. This is consistent with contention not being
an issue.