Home > PR 2004 > Large-Scale Molecular Dynamics

Large-Scale Molecular Dynamics Simulations as a First Step in Understanding Lubrication
Using the ORNL CRAY X1

D. C. Rapaport, Bar-Ilan University and Oak Ridge National Laboratory
M. R. Fahey, Oak Ridge National Laboratory
P. T. Cummings, Vanderbilt University and Oak Ridge National Laboratory
Y. Braiman, Oak Ridge National Laboratory

What does this calculation mean for the future? There is an enormous variety of problems involving MD that require large amounts of computation, both major extensions of existing studies and exploration in new fields that have yet to be tackled because of a lack of adequate high-performance computing capability. This benchmark demonstrates the capability of the Cray X1 to deal with problems of this type. This calculation was performed on 512 processors. The CRAY X1 at ORNL is currently expanded to 2048 processors. About the image Imaging such a complex 3-d system is nontrivial. Here is a slice through a layer of (about 1200) nearly rigid polymer chains. The chains shown in red extend over more than 2 atom diameters in the direction normal to the shear plane; these are in the minority, and most molecules (shown in green) align within the shear plane or very close to it.

Molecular dynamics (MD) is a key methodology for the study of collective phenomena in nanoscience in those situations where the forces between atoms and/or molecules can be modeled by classical force fields. Examples abound, and include self-assembly of organic and inorganic nanostructures, structure and dynamics of nanobiological systems, many aspects of nanotribology, and the properties of polymer nanostructures, to name a few.

As a preliminary test, we implemented vectorization of Rapaport’s scalable domain-decomposed MD code for simple fluids. This simulation demonstrated essentially perfect scaling up to the full size of the Cray X1 (512 MSP nodes) at NCCS. Using this code, we performed brief MD simulations involving up to 12 billion atoms, the largest system permitted by the Cray X1 memory at the time of the runs. The benchmark summary below is for a series of soft-sphere systems at moderate density (0.8). The runs, each over 1000 timesteps, are carried out in SSP (single-streaming) rather than MSP (multi-streaming) mode because of significantly higher performance. The system sizes are chosen to almost fill the memory available; the number of molecules per processor is similar for all runs and the results reveal little variation in overall execution time despite the 250-fold size range. The key performance indicator is the time required per molecule for a single step; this scales inversely as the number of processors, dropping to almost a nanosecond in the final case.

size (molecules)		processors (ssp)		memory (gigabyte)		time/mol-step (nanosec)		run time (min)
47,409,408		8		7		302.1		239.6
379,275,264		64		57		39.5		251.4
1,291,315,456		216		193		12.3		267.5
3,054,207,744		512		458		5.2		268.4
5,988,775,936		1000		894		2.8		285.8
12,093,857,792		2016		1816		1.4		285.0

Performance will improve further as hardware speeds increase and even more massive parallelism becomes available. While several years are likely to elapse before the largest systems can be simulated over millions of timesteps, systems with merely a few million molecules can already be followed over meaningful time periods relatively quickly, fast enough even for real-time visualization. There is no shortage of applications in fields such as material science, polymer fluids and surface science that stand to benefit from simulations of this type.

One of the applications of the MD is calculation of friction between surfaces, an important, longstanding scientific problem. Of particular importance is the issue of thin-film lubrication, which traditionally has received little attention owing to the complexity of the phenomenon. MD simulation is capable of not only reproducing the observed bulk behavior of fluids confined between sheared surfaces under load, but also provides access to the detailed molecular structure and correlations that are actually responsible for this behavior. We are planning to perform simulations of nanotribological systems to resolve several outstanding lubrication questions and controversies. In order to accomplish this goal for polymer-based lubricants that are subject to high strain rates, the simulation of systems containing large numbers of molecules will be required; this is to ensure that the correct spatial and conformational organization of the molecules is able to emerge naturally, free from artifacts due to finite-size effects. Furthermore, the runs must be of sufficiently long duration to capture the time-dependent aspects of the behavior; this is particularly important since the intrinsic relaxation times associated with polymer dynamics increase rapidly with molecular size.

Our computations are able to utilize the vector and parallel capabilities of the Cray X1/X2 computers efficiently. The combination of vectorization to improve single-processor performance and large scale parallelization to permit larger system sizes to be simulated [D. C. Rapaport, "The Art of Molecular Dynamics Simulation", Cambridge, 2004] will be essential to success in this challenge. Analysis of the simulation results will involve both quantitative and qualitative techniques, the latter relying extensively on advanced three-dimensional computer graphics capabilities. While we will look at algorithmic improvements to help in bridging length and time scales, the application of the largest and most powerful supercomputers available will make possible the key calculations needed to understand nanotribological behavior.