DOLIB shared memory for modeling contaminant transport in ground-water

Problem: One of the most common technique for aquifer re-mediation is to pump contaminated water to the surface for treatment. The effectiveness of Pump-And-Treat (PAT) depends highly on the soil heterogeneity. A computational investigation on the effectiveness of PAT re-mediation in heterogeneous aquifers was performed by Dr Mahinthakumar.

The movement of contaminant plume was modeled by a particle tracking scheme. Particle tracking is a also key computational kernel in Lagrangian transport algorithms, molecular simulation and PIC (particle-in-cell) methods. A common technique involves a spatial decomposition of the computational grid with padding (or extended ``ghost cells'') to contain flow field information from neighboring regions. The time-step is then chosen to ensure no particle can escape the extended region. This technique either imposes a very severe constraint on the time step allowed or requires high overhead in memory use and communication volume associated with a large ghost region. Load imbalance is usually a problem since the cost of tracking a particle may depend on the local characteristics of the flow field.

Approach: Fully three-dimensional advection-dispersion effects are modeled. The flow field is stored in a globally accessible array (provided by DOLIB) and interpolated as needed. As a result, particles need not be statically assigned to processors and dynamic redistribution of particles for load balancing is possible. Access to shared memory is through explicit calls to gather and scatter and requires no new language extensions, no compiler nor special operating systems support. Read-only data is cached within DOLIB for good performance. DOLIB also provides atomic accumulate operations (without explicit locks) and generalized test-and-set operation for implementing dynamic load balancing.

Progress and Results: A large scale simulation was performed on 32nodes of an Intel Paragon. A total 440K particles with 500 time steps using a velocity field pre-computed from a flow code. DOLIB is used in the velocity mapping and concentration computation stages. The initial concentration is show be be reduced by Pump-And-Treat re-mediation to a much smaller plume after 1000 days of treatment. Table 1 shows the run time for each module.


efdazedo@msr.EPM.ORNL.GOV
Tue Jan 17 11:35:28 EST 1995