|
|
|
|
The Cheetah system at Oak Ridge National Laboratory is comprised of 24 p690 nodes. Each node has 32 processors. Two of the nodes have 128GB of memory, two have 64GB, while the rest have 32GB. Eight of the 32GB nodes are partitioned into four 8-processor/8GB "virtual" (LPAR) nodes, for a total of 32 8-processor nodes. As each node will have a maximum of two interfaces into the network when we install the Colony switch, the use of LPAR nodes will increase network bandwidth available to each processor in a partitioned p690 by a factor of four over that of a processor in an unpartitioned node.The four types of nodes (32-processor/128GB, 32-processor/64GB, 32-processor/32GB, 8-processor/8GB LPAR) have somewhat different memory subsystem characteristics. For example, the 128GB nodes use dual-ported memory devices, while the 32GB nodes use single-ported memory. The LPAR nodes partition memory as well as processors, and each runs its own OS, which also impacts memory usage. To examine the practical impact of these different configurations, we ran a series of experiments in February, 2002 using the PSTSWM serial benchmark code and the CCM/2D-MP and PCTM climate applications. The introduction of large page support and more sophisticated process/memory placement algorithms will require rerunning these experiments in late Spring of 2002.