The main goal of this lecture is to address the numerical simulation of a phenomenon which has been observed experimentally, namely: the clustering of particles in sub-populations of essentially equal size when a circular cylinder containing a mixture rigid solid particles / incompressible viscous fluid spins with a sufficiently large angular velocity. The numerical simulator relies on the combination of (i) a time discretization by operator-splitting; (ii) a fictitious domain method allowing a uniform mesh based finite element approximation (of the Bercovier-Pironneau type) in three space dimensions. The results of our numerical simulations are consistent with the experimental ones and show that, indeed, particle clustering takes place when the spinning angular velocity n is sufficiently large.

To provide true predictive utility, subsurface simulations often must accurately resolve--in three dimensions--complicated, multi-phase flow fields in highly heterogeneous geology with numerous chemical species and complex chemistry. This task is especially daunting because of the wide range of spatial scales involved--from the pore scale to the field scale--ranging over six orders of magnitude, and the wide range of time scales ranging from seconds or less to millions of years. This represents a true "Grand Challenge" computational problem, requiring not only the largest-scale ("ultrascale") supercomputers, but accompanying advances in algorithms for the efficient numerical solution of systems of PDEs using these machines, and in mathematical modeling techniques that can adequately capture the truly multi-scale nature of these problems.

In this talk, Richard will describe some of the specific challenges involved in simulating subsurface reactive flows using ultrascale supercomputers and discuss how they are being addressed in the development of the computer code PFLOTRAN. Richard will begin with some brief background material on the governing equations of subsurface reactive flows and on modern parallel computer architectures and programming models. RichardI will then discuss the software and algorithmic approaches that have enabled PFLOTRAN to provide scalable parallel performance on many thousands of processors. I will describe ongoing work to address disparate time and spatial scales by both the development of adaptive mesh refinement methods and the use of multiple continuum formulations. Finally, Richard will present some examples from recent simulations conducted on Jaguar, the 23412 processor core CrayXT3/4 system at Oak Ridge National Laboratory that is currently one of the top ten fastest supercomputers in the world.

This presentation will be in three parts. In part (I), stabilized finite element and finite volume methods for fully incompressible and compressible flows will be introduced. The parallel scalability and vector computations of CaMEL flow solvers will be discussed. In part (II), a hybrid approach based on both finite element and finite volume methods for incompressible flows will be introduced.

The finite element applications will include the dispersion of contaminants in urban and battlefield environments, and free surface flows in hydrodynamics of ships in motion. The motion of the ships is accounted by coupling the nonlinear 6DOF rigid body dynamics with Navier–Stokes equations. The mesh-moving algorithms are based on linear and nonlinear elasticity equations. The sustained computational speed is measured more than one teraflop using unstructured meshes with two billion tetrahedral elements using Cray X1E with 256 processors. Many 3D numerical examples will be presented.

This presentation will report the performance of CaMEL_Aero, our truly matrix-free, parallel and vectorized unstructured finite volume solver for compressible flows. The Jacobian-free Generalized Minimal RESidual (GMRES) method is used to solve the nonlinear systems of equations using Newton-Raphson iterations. Furthermore, the matrix-free Lower-Upper Symmetric Gauss Seidel (LU-SGS) method is employed as a preconditioning technique to the GMRES solver. The aerodynamics calculation of complex 3D geometries will be discussed. This includes steady-state and transient aerodynamics calculations for Formula-1 racing cars as well as supersonic spinning projectiles.

In part (III), the development of an implicit hybrid (finite element and finite volume) flow solver for the incompressible Navier-Stokes equations will be introduced. The methodology is based on the pressure correction or projection method. A fractional step approach is used to obtain an intermediate velocity field by solving the original momentum equations with the matrix-free implicit cell-centered finite volume method. The Poisson equation derived from the fractional step approach is solved by the node-based Galerkin finite element method for an auxiliary variable. The auxiliary variable is closely related to the real pressure and is used to update the velocity field and the pressure field. We store the velocity components at cell centers and the auxiliary variable at cell vertices, making the current solver a staggered-mesh scheme. Several examples will be provided.

In this talk, we target a class of mixed variable optimization problems in which the objective and constraint functions may be nonsmooth, discontinuous, extended valued, computationally expensive to evaluate, given as a black box, provide few significant digits of accuracy, and may even fail to return a value. Furthermore, constraint functions may return only a yes/no value, derivatives are generally unavailable, and variables may be a mix of continuous, integer, or nonnumeric categorical types (such as color, shape, or type of material). Our approach for solving problems of this type is the derivative-free class of mesh adaptive direct search algorithms. We describe the details of this algorithm class, along with their first-order and second-order theoretical convergence properties, and then introduce some scientific and engineering applications for which satisfying numerical results have been obtained.

Clusters of galaxies are the largest bound objects in the Universe, and most of their normal matter exists in the form of extremely hot, diffuse, gas. High-resolution X-ray observations have revealed cavities with sharp edges in temperature, density, and metallicity in this gas; Their presence poses a puzzle since these features are not expected to be hydrodynamically stable, or to remain sharp in the presence of diffusion.

The mergers of galaxy clusters can be the most energetic events in the Universe, and even `minor' mergers are very dynamic events, which eventually lead to the stripping and dissolution of the smaller core. Where this occurs is important -- it helps determine the thermal and compositional history of the evolution of the cluster (and thus its component galaxies).

With large 3D adaptive-mesh MHD simulations, and smaller more focused 2D simulations, we show how a moving core or bubble in even a very weakly magnetized plasma necessarily sweeps up enough magnetic field to build up a dynamically important sheath around the object; the layer's strength is set by a competition between `plowing up' of field and field lines slipping around the core, and to first order depends only on the ram pressure seen by the moving object.

Despite the spectacular progress made in high-performance communication interfaces, many potential optimizations are still unexplored. It is mainly due to efforts in shortening the critical path to the maximum, in order to minimize latency. NewMadeleine is a communication library built around a powerful transfer optimizing engine with generic optimization strategies. These generic strategies are parametrized by the capacities of the target underlying network driver. By coordinating its own activity with that of the network cards, NewMadeleine has a global view of the application communication flows. Then, it becomes easy to multiplex them onto one or more physical network cards, even in cases of heterogeneous technologies. Moreover, the current trend in clusters leads towards an increase of the number of cores per node. In order to fully exploit them, NewMadeleine works in collaboration with PIOMan, the PM2 I/O Manager. PIOMan is a multithreaded communication engine able to exploit idle cores to speed up communications. We will thus see how this association takes advantage of all the available resources, whether in terms of computation or communication units, in order to improve the global application execution time.

Élisabeth Brunet, a PhD student, has been a member of the INRIA Runtime team of LaBRI (University of Bordeaux, France) since october 2005 (http://www.labri.fr/perso/brunet/). Her primary research interests are high-performance networking, generic communications algorithms for high-speed networks, and communication middleware programming. She is one of the principal designers of the NewMadeleine high-performance communication library.