New exascale system for earth simulation

April 23, 2018
Article Image

The high-resolution E3SM earth system model simulates the strongest storms with surface winds exceeding 150 mph—hurricanes that leave cold wakes that are 2 to 4 degrees Celsius cooler than their surroundings. This simulation represents how sea surface temperature changes evolve as a hurricane (seen here approaching the U.S. East Coast) moves across the Atlantic and how the resultant cold wake affects subsequent intensification of the next hurricane. (Download Image)

New exascale system for earth simulation

Anne M Stark, stark8@llnl.gov, 925-422-9799

A new earth modeling system unveiled today will have weather-scale resolution and use advanced computers to simulate aspects of Earth’s variability and anticipate decadal changes that will critically impact the U.S. energy sector in coming years.

After four years of development, the Energy Exascale Earth System Model (E3SM) will be released to the broader scientific community this month. The E3SM project is supported by the Department of Energy's Office of Science in the Biological and Environmental Research Office. The E3SM release will include model code and documentation, as well as output from an initial set of benchmark simulations.

The Earth, with its myriad interactions of atmosphere, oceans, land and ice components, presents an extraordinarily complex system for investigation. Earth system simulation involves solving approximations of physical, chemical and biological governing equations on spatial grids at resolutions that are as fine in scale as computing resources will allow.

The E3SM project will reliably simulate aspects of earth system variability and project decadal changes that will critically impact the U.S. energy sector in the near future. These critical factors include a) regional air/water temperatures, which can strain energy grids; b) water availability, which affects power plant operations; c) extreme water-cycle events (e.g. floods and droughts), which impact infrastructure and bio-energy; and d) sea-level rise and coastal flooding, which threaten coastal infrastructure.

E3SM will provide insights on earth system interactions in the Arctic and their influence on mid-latitude weather. In this E3SM model simulation, winter storm clouds, represented here by outgoing longwave radiation, or OLR, affect sea ice coverage as the clouds move across the Arctic.

The goal of the project is to develop an earth system model (ESM) that has not been possible because of limitations in current computing technologies. Meeting this goal will require advances on three frontiers: 1) better resolving earth system processes through a strategic combination of developing new processes in the model, increased model resolution and enhanced computational performance; 2) representing more realistically the two-way interactions between human activities and natural processes, especially where these interactions affect U.S. energy needs; and 3) ensemble modeling to quantify uncertainty of model simulations and projections.

"The quality and quantity of observations really makes us constrain the models,” said David Bader, Lawrence Livermore National Laboratory (LLNL) scientist and lead of the E3SM project. “With the new system, we’ll be able to more realistically simulate the present, which gives us more confidence to simulate the future.”

Simulating atmospheric and oceanic fluid dynamics with fine spatial resolution is especially challenging for ESMs.

The E3SM project is positioned on the forefront of this research challenge, acting on behalf of an international ESM effort. Increasing the number of earth system days simulated per day of computing time is a prerequisite for achieving the E3SM project goal. It also is important for E3SM to effectively use the diverse computer architectures that the DOE Advanced Scientific Computing Research (ASCR) Office procures to be prepared for the uncertain future of next-generation machines.

 

A long-term aim of the E3SM project is to use exascale machines to be procured over the next five years. The development of the E3SM is proceeding in tandem with the Exascale Computing Initiative (ECI) (An exascale refers to a computing system capable of carrying out a billion billion (1018) calculations per second. This represents a thousand-fold increase in performance over that of the most advanced computers from a decade ago.

“This model adds a much more complete representation between interactions of the energy system and the earth system,” Bader said. “The increase in computing power allows us to add more detail to processes and interactions that results in more accurate and useful simulations than previous models.”

To address the diverse critical factors impacting the U.S. energy sector, the E3SM project is dedicated to answering three overarching scientific questions that drive its numerical experimentation initiatives:

  • Water Cycle: How does the hydrological cycle interact with the rest of the human-earth system on local to global scales to determine water availability and water cycle extremes?
  • Biogeochemistry: How do biogeochemical cycles interact with other earth system components to influence the energy sector?
  • Cryosphere Systems: How do rapid changes in cryosphere (continental and ocean ice) systems evolve with the earth system, and contribute to sea-level rise and increased coastal vulnerability?
This simulation depicts the speed of the ocean currents near the surface.

In the E3SM, all model components (atmosphere, ocean, land, ice) are able to employ variable resolution to focus computing power on fine-scale processes in regions of particular interest. This is implemented using advanced mesh-designs that smoothly taper the grid-scale from the coarser outer region to the more refined region.

The E3SM project includes more than 100 scientists and software engineers at multiple DOE laboratories as well as several universities; the DOE laboratories include Argonne, Brookhaven, LLNL, Lawrence Berkeley, Los Alamos, Oak Ridge, Pacific Northwest and Sandia national laboratories. In recognition of unifying the DOE earth system modeling community to perform high-resolution coupled simulations, the E3SM executive committee was awarded the Secretary of Energy's Achievement Award in 2015.

In addition, the E3SM project benefits from-DOE programmatic collaborations including the Exascale Computing Project (ECP) and programs in Scientific Discovery Through Advanced Computing (SciDAC), Climate Model Development and Validation (CMDV), Atmospheric Radiation Measurement (ARM), Program for Climate Model Diagnosis and Intercomparison (PCMDI), International Land Model Benchmarking Project (iLAMB), Community Earth System Model Community Earth System Model (CESM) and Next Generation Ecosystem Experiments (NGEE) for the Arctic and the Tropics.

For information, go the E3SM website.

 

Anne M Stark
925-422-9799