Computational Earth Sciences
The Computational Earth Sciences group advances and communicates the predictive understanding of Earth Systems by developing and applying models, quantitative methods, and computational tools that exploit high performance computing resources.
Oak Ridge National Laboratory (ORNL) is one of eight Department of Energy (DOE) laboratories that will use high-performance computing (HPC) to develop the most sophisticated Earth system model to date for climate change research with scientific and energy applications. The national labs are collaborating with the National Center for Atmospheric Research, four academic institutions, and one private-sector company on this long-term project, known as Accelerated Climate Modeling for Energy, or ACME. Many of the ORNL team members are also Climate Change Science Institute (CCSI) researchers.
DOE and ORNL have been major drivers of Earth system models in recent decades, and ACME will provide scientists with Earth system models that take advantage of upcoming milestones in computing capability. As new HPC architectures support computing power at hundreds of petaflops and then exaflops (a thousand petaflops), more expansive simulations will enable finer climate predictions.
And with more computing power comes significant expectations for scientific discovery.
"The model we are building will provide rapid science advances by including new processes such as phosphorous cycling in tropical ecosystems and by making it easier for domain science experts to participate in model development and application," said Peter Thornton, ORNL land modeler and ACME council member and Land Model Task Team leader. "Through innovative software architecture and intensive collaboration, we're building new science bridges to existing communities of observational and experimental expertise, which will challenge and ultimately improve our ability to predict future climate system dynamics."
Using DOE Office of Science Leadership Computing Facility resources, including ORNL's 27-petaflop Titan supercomputer, the team will set out on a 10-year plan to develop model codes that address key climate science questions, including those related to the water cycle, biogeochemistry, and cryosphere.
In the near term the team will simulate changes in river flow and other parts of the hydrologic cycle by modeling interactions between precipitation and the landscape within high-resolution, fully coupled atmosphere and land surface models. Over the next decade these models will help answer how the water cycle will evolve in a warmer climate and change land and water use.
Likewise, ACME models will explore fundamental questions about the impact of carbon, nitrogen, and phosphorus cycles on the climate system and then simulate changes in a warmer environment once new and developing models have been validated. Simulations of the cryosphere, or surface ice in the form of the continental Antarctic ice sheet, will also gain new depth and resolution, allowing researchers to study its stability in the climate system and the potential effects of sea level rise due to melting.
To contribute to this advanced model development, CCSI experts in terrestrial biogeochemistry and atmospheric chemistry will work with computational scientists to optimize workflow and engineer new software tools for calculating an increased number of scientific variables at higher resolutions on Titan, the Mira supercomputer at Argonne National Laboratory, and other DOE computing resources.
"Our goal is to build the best possible model to run on DOE computers, and with an optimized workflow and a strong software engineering infrastructure, we'll be able to broaden access for domain scientists to tackle detailed scientific problems," Thornton said.
CCSI and ORNL researchers involved in the program leadership are atmospheric scientist Kate Evans as Workflow Task Team colead, computer scientist Patrick Worley as Performance Task Team leader, and ORNL National Center for Computational Sciences Director James Hack as ex officio council member.
More information can be found in the Accelerated Climate Modeling for Energy: Project Strategy and Initial Implementation Plan - edited by Katie Elyce Jones
Regional Modeling Frameworks (site PI, Moet Ashfaq)
Predicting the regional hydrologic cycle at time scales from seasons to centuries is one of the most challenging goals of climate modeling. Because hydrologic cycle processes are inherently multi-scale, increasing model resolution to more explicitly represent finer scale processes may be a key to improving simulations of the hydrologic cycle. The overall objective of the proposed research is to develop frameworks for robust modeling of regional climate and hydrologic cycle, and to improve understanding of factors contributing to uncertainties in simulating future changes in the regional hydrologic cycle. We propose a hierarchical approach to test the veracity of global high resolution, global variable resolution, and nested regional climate model for regional climate modeling. We hypothesize that hierarchical evaluation of different modeling approaches will lead to better understanding of their relative merits and improve the frameworks for robust regional climate simulation. Our evaluation hierarchy has four stages: 1) Idealized, no physics test cases, 2) Idealized, full physics test cases, 3) Real world, atmosphere-only and ocean-only simulations, and 4) Real world, coupled atmosphere-ocean simulations for both current and future climate.
At each stage, four types of experiments will be performed:
- Global Simulations at High Resolution (GS-HR)
- Global Simulations at Low Resolution (GS-LR)
- Global Simulations using Variable Resolution (GS-VR) with high resolution in the area of interest and low resolution elsewhere
- Regional Simulations using High Resolution (RS-HR) within a limited-area domain with the lateral boundary forcing provided by GS-LR.
The GS-HR and GS-LR simulations will be performed using CCSM with three different dynamical cores—Spectral, HOMME, and Model for Prediction Across Scales (MPAS). The latter can be configured for GS-HR, GS-VR, and RS-HR so all three modeling approaches can be compared within a single framework. The RS-HR simulations will be performed using MPAS, WRF, and RegCM. For GS-VR and RS-HR, the high-resolution regions will be located in North and South America where the regional hydrologic cycle exemplifies scale interactions and atmosphere-land-ocean feedbacks that challenge regional climate modeling.
The global and regional simulations will be compared and evaluated to assess (a) the impacts of different dynamical cores for global high-resolution simulations, (b) multiple techniques for mesh refinement, (c) the upscaled impacts of the high resolution region, and (d) the overall value of regional climate simulation. We will also apply regional and global diagnostics, evaluation metrics, and process-based analysis to the simulations to determine (1) whether modeling frameworks that allow scale interactions through global high resolution or variable resolution may be more skillful in simulating the regional hydrologic cycle in climate regimes dominated by convection; (2) whether models that couple atmosphere and ocean at the regional scale are more skillful in simulating regional climate variability in the west coast of North and South America; and (3) whether differences in simulating feedbacks by different modeling approaches may be modulated by surface heterogeneities to amplify differences in simulating regional hydrologic cycle changes in the future climate.
The BER SciDAC Partnerships efforts focus on two main research thrusts: climate and environmental sciences and biological systems. Partnerships in climate and environmental sciences aim to advance the simulation and predictive capabilities of state-of-science climate modeling and provide improved models for better understanding the movement of subsurface contamination. Partnerships in biological systems seek to develop new methods for modeling complex biological systems, including molecular complexes, metabolic and signaling pathways, individual cells and, ultimately, interacting organisms and ecosystems.
Multiscale Project (Jim Hack, site PI)
The MULTISCALE project’s primary goal is to produce better models for these critical processes and constituents, from ocean-eddy and cloud-system to global scales, through improved physical and computational implementations. An integrated team of climate and computational scientists will accelerate the development and integration of multiscale atmospheric and oceanic parameterizations into the DOE-NSF Community Earth System Model (CESM). The team’s technical objective is to introduce accurate and computationally efficient treatments of interactive clouds, convection, and eddies into the next generation of CESM at resolutions approaching the characteristic scales of these structures. The project delivers treatments of these processes and constituents, which are scientifically useful over resolutions ranging from 2 to 1/16 degrees.
The MULTISCALE team will develop, validate, and apply multiscale models of the climate system based upon atmospheric and oceanic components with variable resolution. The project will exploit new variable-resolution unstructured grids based on finite-element and finite-volume formulations developed by team members. Effective deployment of these dynamical cores will require significant and concurrent advances in time-stepping methods, grid generation, and automated optimization methods for next-generation computer architectures.
Predicting Ice Sheet and Climate Evolution at Extreme Scales (PISCEES)(Kate Evans, site PI)
Although continental scale ice sheet models have improved in recent years, much work is needed to make these models reliable and efficient on continental scales and to quantify their uncertainties. We therefore propose to develop and apply robust, accurate, and scalable dynamical cores ("dycores") for ice sheet modeling on structured and unstructured meshes with adaptive refinements; to evaluate ice sheet models using new tools and data sets for verification and validation (V&V) and uncertainty quantification (UQ); and to integrate these models and tools in the Community Ice Sheet Model (CISM) and with coupled Earth system models, including the Accelerated Climate Model for Energy (ACME) and the Community Earth System Model (CESM). We are developing tools and frameworks for V&V, including verification test suites consisting of analytical and manufactured solutions, and we will compile the best available data sets for model validation. These tools will be assembled in a post-processing package that can be used to formally evaluate CISM in standalone runs and as a CESM component. PISCEES will provide a coherent structure for ongoing collaboration among glaciologists, climate modelers, and computational scientists. We will work closely with the SciDAC institutes (FASTMath, QUEST, and SUPER) and the community of model developers.
NGEE Arctic (CESG contact: Forrest Hoffman)
NGEE tropics (CESG contact: Forrest Hoffman)
EERE (CSEG contact: Moet Ashfaq)
Panorama (CESG contact: Ben Mayer)
As we move closer toward the ability to execute exascale calculations and process the ensuing extreme amounts of data produced by both experiments and computations alike, the complexity of managing the compute, data analysis and data management tasks has grown beyond the capabilities of domain scientists. Thus, workflow management systems are required to ensure current and future scientific discoveries.
Impacts of Aerosols and Air-Sea Interaction on Community Earth System Model (CESM) Biases in the Western Pacific Warm Pool Region (contact Mahajan)
The realistic simulation of clouds and precipitation over the Tropical Western Pacific warm pool (WPWP) region remains a challenge in this new era of higher resolution climate models, where air-sea coupling plays a strong role. The region also bears an influx of aerosols from Southeast Asia along with local sources from the Western Pacific islands, which can play an important role in the regional climate. The relationship between aerosols and clouds/precipitation is non-monotonic, because of the simultaneous direct, semi-direct and indirect effects of aerosols on the surface, clouds and precipitation. The microphysical interactions between aerosols and clouds have only recently been incorporated into global climate model (GCM) simulations. It is important to understand these simulated interactions and compare them with observations to improve the representation of climate processes in GCMs and hence improve climate predictions. We are conducting a hierarchical coupled modeling study using a suite of experiments with the Community Earth System Model (CEMS1.0), comparing model output with observations from DOE ARM surface instruments as well as multi-sensor satellite observations. This hierarchy of mechanistic integrations will facilitate the understanding of the interaction between aerosols and the atmospheric column, as well as the interaction between the atmospheric column and the underlying ocean, in determining the tropical climate state. Insights gained from analyzing these integrations can be used to improve model parameterization of aerosol effects, and thus help alleviate model biases associated with clouds and precipitation.
Large-Scale Organization and Scale-Interaction: A Dynamical Pathway toward Understanding, Modeling and Predicting Energy and Water Cycle Extremes (contact: Kate Evans)
Extreme weather events (EWEs) related to variations in the energy and water cycles have important economic and societal consequences. The project is a collaboration between Georgia Tech and DOE's ORNL that addresses the knowledge shortcomings in relation to DOE RGCM research priorities. We will study the following energy and water cycle extremes: winter cold air outbreaks and warm waves, summer heat waves, and droughts and floods. The research will concentrate on revealing the dynamical pathway for the large-scale organization and scale interaction that bridges planetary-scale climate modes (PCMs), large-scale meteorological patterns (LMPs), and EWEs. The initial part of our research will employ atmospheric observational data and focus on (a) objectively characterizing LMPs associated with different classes of EWEs and (b) developing simple and concise atmospheric circulation metrics for identifying such LMPs in gridded datasets. We expect that atmospheric blocking will play a critical role in several EWE classes. Secondly, our project consists of a detailed diagnosis of the physical mechanisms by which (a) the implicated LMPs arise, (b) LMPs enact EWEs and (c) natural modes of climate variability (i.e., PCMs) serve to modulate LMPs/EWEs. The diagnostic results will be used to construct a set of dynamical metrics and incorporate them into a formal evaluation process to be used in DOE model development efforts. The third part of our project is the parallel study of EWEs, LMPs and PCMs in simulations of historical and future climate. There is particular interest in studying the benefits gained through enhanced model resolution by examining coupled model simulations conducted at ORNL.
Forwarn (CESG contact: Hoffman)
SUPER: (CESG contact: Patrick Worley)
The Institute for Sustained Performance, Energy, and Resilience (SUPER) project is a broadly-based SciDAC (Scientific Discovery through Advanced Computing) Institute with expertise in compilers and other system tools, performance engineering, energy management, and resilience. The goal of the project is to ensure that Department of Energy (DOE) computational scientists can successfully exploit the emerging generation of high performance computing (HPC) systems. These HPC systems are evolving significantly from those in use in the recent past: concurrency is scaling exponentially; accelerators such as graphics processing units (GPUs) are being utilized; and even the memory hierarchy may change with the incorporation of a new generation of persistent devices (e.g., phase change memory). The goal is being met by providing application scientists with strategies and tools to productively maximize performance, conserve energy, and attain resilience.
EPSi (CESG contact: Patrick Worley)
The EPSI SciDAC (Sustained Performance, Energy, and Resilience ) Science Application Partnership project is developing advanced simulation software, utilizing extreme parallelism and based upon a first-principles kinetic approach, to address the challenges associated with understanding the edge region of magnetically confined plasmas. The plasma edge presents a set of multi-physics, multi-scale problems involving a separatrix and complex 3D magnetic geometry. Perhaps the greatest computational challenge is the lack of scale separation - temporal scales for drift waves, Alfvén waves, and ELM dynamics, for example, have strong overlap. Collaborations with the SciDAC Institutes ensure reliable and efficient solution of validated models on extreme-scale, high-performance computers.
ACES2BCG (Forrest Hoffman, PI)
The ACES4BGC Project seeks to advance the predictive capabilities of Earth System Models (ESMs) by reducing two of the largest sources of uncertainty, aerosols and biospheric feedbacks, with a highly efficient computational approach. In particular, this project will implement and optimize new computationally efficient tracer advection algorithms for large numbers of tracer species; add important biogeochemical interactions between the atmosphere, land, and ocean models; and apply uncertainty quantification (UQ) techniques to constrain process parameters and evaluate uncertainties in feedbacks between biogeochemical cycles and the climate system. The resulting improvements to the Community Earth System Model (CESM)will deliver new scientific capabilities and significantly improve the representation of biogeochemical interactions at the canopy-to-atmosphere, rivers-to-coastal oceans, and open oceans-to-atmosphere interfaces. ACES4BGC partners modelers with decades of cumulative research experience and a team of computer and computational scientist building scalable solvers and tools, developing advanced UQ methods, and applying technologies for performance optimization through U.S. DOE SciDAC Institutes. The project began April 15, 2012.
- The current Earth System Modeling projects are sponsored by the DOE Office of Science SciDAC Program. This program is titled Scientific Discovery through Advanced Computing, is exploring and developing the third leg of modern science - simulation - as a paradigm complementing experiment and theory. Funding for these projects is primarily for model development and comes from the DOE Office of Biological and Environmental Research, Climate Research Division.
- The application of the models falls into the applied earth sciences area and takes on a variety of forms requiring specialized techniques of data assimilation, and data analysis. Our sponsors for this area are NASA Earth Missions as well as DOE OBER.
- Mathematical aspects of our modeling work are partially sponsored by the Office of Advanced Scientific Computing Research
- Work on GCM Downscaling and Economic Impacts is supported by the ORNL LDRD program.
Selected Recent Publications from Group Members and affiliates:
Climate Science Performance, Data and Productivity on Titan. Cray User Group Meetin 2015. Benjamin Mayer, Patrick H. Worley, Rafael Ferreira da Silva, Abigail L. Gaddis
PANORAMA: An Approach to Performance Modeling and Diagnosis of Extreme Scale Workflows. International Journal of High Performance Computing Applications. Deelman, Ewa ISI Christopher D. Carothers, Prof. Rensselaer Polytechnic Institute (RPI) Anirban Mandal, University of North Carolina, Renaissance Computing Insitute (RENCI), Chapel Hill Brian Tierney, University of California/Lawrence Berkeley National Laboratory (LBNL) Jeffrey S. Vetter, ORNL (906493) Ilya Baldin, University of North Carolina, Renaissance Computing Insitute (RENCI), Chapel Hill Claris Castillo, University of North Carolina, Renaissance Computing Insitute (RENCI), Chapel Hill Gideon Juve, ISI Dariusz Krol, ISI Vickie E. Lynch, ORNL Benjamin W. Mayer, ORNL Jeremy S. Meredith, ORNL Thomas E. Proffen, ORNL Paul Ruth, University of North Carolina, Renaissance Computing Insitute (RENCI), Chapel Hill Rafael Ferreira Da Silva, ISI
W. D. Collins, H. Johansen, K. J. Evans, C. Woodward, and P. Caldwell (2015). Progress in Fast, Accurate Multi-scale Climate Simulations, Procedia Computer Science, accepted.
Mahajan S., K. J. Evans, M. Branstetter, V. Anantharaj and J. K. Leifeld (2015): Fidelity of precipitation extremes in high resolution global climate simulations, Procedia Computer Science, accepted.
Rui, M. M. Ashfaq, D. Rastogi, L. R. Leung, and F. Dominguez (2015), Dominating Controls for Wetter South Asian Summer Monsoon in the 21st Century, J. Climate, 28, 3400–3419.
P. Parham, J. Waldock, D. Hemming, K. Evans et al. (2015). Climate, Environmental, and Socioeconomic Change - Weighing up the Balance in Vector-Bourne Disease Transmission. Phil. Trans. B , 370:20130551. doi: 10.1098/rstb.2013.0551.
Touma, D., M. Ashfaq, M. A. Nayak, S. C. Kao, and N. S. Diffenbaugh (2014), A Multi-model and Multi-index Evaluation of Drought Characteristics in the 21st Century, Journal of Hydrology, 526, 196-207. doi:10.1016/j.jhydrol.2014.12.011
Kapnick, S. B., T. L. Delworth, M. Ashfaq, S. Malyshev, and P. C. D. Milly (2014), Snowfall less sensitive to warming in Karakoram than in Himalayas due to a unique seasonal cycle, Nature Geoscience, 7, 834–840.
K. J. Evans, S. Mahajan, M. Branstetter, J. McClean, J. Caron, M. Maltrud, J. Hack, D. Bader, R. Neale, J. Leifeld (2014). A spectral transform dynamical core option within the Community Atmosphere Model, version 4. J. Adv. Mod. Earth Sys.6:902-922. doi: 10.1002/2014MS000329.
T. Jiang, K. J. Evans, Y. Deng, X. Dong (2014). Intermediate Frequency Atmospheric
Disturbances: A Dynamical Bridge Connecting western U.S. Extreme Precipitation with East Asian Cold Surges in linking atmospheric weather extremes. JGR: Atmospheres , 119:3723-3735. doi: 10.1002/2013JD021209.
R. Langan, R. Archibald, M. Plumlee, S. Mahajan, D. Ricciuto, C. Yang, R. Mei, J. Mao, X. Shi and J. Fu (2014): Stochastic Parameterization to Represent Variability and Extremes in Climate Modeling, Procedia Computer Science, 29, 1146-1155
Lu, D., M. Ye, M. C. Hill, E. P. Poeter, and G. P. Curtis (2014), A Computer Program for Uncertainty Analysis Integrating Regression and Bayesian Methods, Environmental Modeling & Software, Vol. 60, pp.4156.
Ashfaq M., S. Ghosh, S-C Kao, L. C. Bowling, P. Mote, D. Touma, S. A. Rauscher and N. S. Diffenbaugh (2013), Near-term acceleration of hydroclimatic change in the western U.S., Journal of Geophysical Research, doi: 10.1002/jgrd.50816
Diffenbaugh, N. S., M. Scherer and M. Ashfaq (2013), Continued global warming intensifies snow-dependent hydrologic extremes in the northern hemisphere, Nature Climate Change, 3(4), 379-384.
S. Mahajan, K. J. Evans, J. Truesdale, J. Hack, (2013). Linearity of climate response to increases in Black Carbon Aerosols. J. Climate , 26 :8223-8237. doi: 10.1175/JCLI-D-12-00715.1.
Lu, D., M. Ye, P. D. Meyer, G. P. Curtis, X. Shi, X. Niu, and S. B. Yabusaki (2013), Effects of Error Covariance Structure on Estimation of Model Averaging Weights and Predictive Performance, Water Resources Research, Vol. 49(9), pp. 60296047.
Group Leader: Kate Evans
Administrative Assistant: Teresa Hurt
- Moet Ashfaq (firstname.lastname@example.org) - Climate variability and change
- Marcia Branstetter (email@example.com) - Hydrology and river flows
- Nate Collier (firstname.lastname@example.org) -
- Kate Evans (email@example.com) - Atmospheric dynamics and numerical methods
- Forrest Hoffman (firstname.lastname@example.org) - Land and carbon modeling
- Salil Mahajan (email@example.com) - Climate variability and dynamics
- Ben Mayer (firstname.lastname@example.org) - Workflow optimization and automation
- Patrick Worley (email@example.com) - Parallel algorithms, performance engineering, and numerical methods
- Min Xu (firstname.lastname@example.org) -
Postdocs and Students
- Abigail Gaddis
- Tianyu Jiang
- Joe Kennedy
- Mahesh Kovilakam
- Roisin Langan
- Dan Lu
- Rui Mei
- Deeksha Rastogi
- Cheng-en Yang
- Valentine Anantharaj - model optimization and analysis
- Rick Archibald - numerical methods and model evaluation
- Joshua Fu (UTK Civil and Environmental Engineering, email@example.com) - Dynamic Downscaling for Climate Impacts on Air Quality and Hydrology
- Bill Hargrove (visiting scientist) - Landscape ecology
- Jitendra Kumar - Hydrology and computational methods
- G. Mahinthakumar (firstname.lastname@example.org) - Hydrology and performance modeling
- Matt Norman - Numerical methods and optimizing for hybrid HPC architectures
- Duane Rosenberg