Transforming the future: LLNL team explores electric grid modernization via HPC

A Lawrence Livermore National Laboratory (LLNL) team has successfully deployed a widely used power distribution grid simulation software on a high-performance computing (HPC) system, demonstrating substantial speedups and taking a key step toward creating a commercial tool that utilities could use to modernize the grid.

Partnering with power management company Eaton to use its CYME software on LLNL’s HPC Quartz system, researchers were able to run thousands of simulations in parallel across 100 core computations. This was accomplished faster than previously possible, which could enable utility companies to better plan energy distribution and simulate the impact of renewable energy resources on the entire electrical grid.

"Essentially, we showed we can run 100 circuits in parallel, and it will take the same time as a single circuit," LLNL researcher Vaibhav Donde said. "But 100 is not a magic number. We have much more capability than that at the Lab, so we can run thousands or maybe more of those scenarios in parallel and achieve a similar order of performance. It opens up the possibility of what we can do in the future and may transform how utilities plan the future grid."

A project of this scale always poses a set of unique challenges, researchers said. It required creative solutions by Livermore Computing HPC and Engineering Software Toolkit groups to host various elements of the CYME software.

The results mean that a utility company as large as Southern California Edison, for example, would be able to run simulations on the entire grid and multiple scenarios at the same time using existing computational techniques and high-performance computing, which hasn’t been achieved before, Donde said.

For California and Hawaii to reach state mandates of 100 percent renewable energy by 2045, Donde said, utility companies will need to run various renewable energy scenarios on a utility-scale distribution network. This work will require the co-simulation and analysis of transmission and distribution systems, which has been largely infeasible due to the challenges in computational scaling and lack of computational resources.

The successful demonstration marks the first phase of a Department of Energy Technology Commercialization Fund (TCF) project focusing on implementing power transmission and distribution grid simulation in a co-simulation framework. The goal is to take the software to market for utility companies, enabling them to see how various, widespread renewable energies such as solar or wind power might impact the overall electrical grid. The two-year project began in late 2017.

"Utilities have been looking for more advanced planning tools to help them better evaluate the impact of distributed energy resources on the operation of bulk systems," said Eaton Corporation’s Senior Specialist and Grid Intelligence Theme Leader Wei Ren, who is providing technical support and managing the project for Eaton. "By pairing Eaton’s CYME software with the newest simulation/computation technology developed by LLNL, utilities can conduct hosting capacity analysis and system impact studies with greater accuracy and speed, driving greater adoption of renewables in power systems. We see our collaboration with LLNL as providing a key advantage for utility customers."

The grid industry traditionally has modeled transmission and distribution grids separately because electricity has historically flowed in one direction, from transmission lines to consumers. But as renewables are increasingly integrated into the greater grid, simulating the transmission and distribution grids together has become essential to predicting reliability and safety.

Building on the computational enhancement the team achieved with CYME, the team will move on to performing transmission and distribution co-simulation using HELICS, a tool developed by the Department of Energy’s Grid Modernization Lab Consortium (GMLC), as well as the electrical power transmission grid simulator GridDyn, developed by LLNL. It will be the first time HELICS will be used with a commercial distribution grid simulation software in a co-simulation environment, researchers said.

"We will see significant increasing deployment of distributed energy resources at distribution system such as solar photovoltaics, demand response and electric vehicles, as well as new sensors, controls and communication networks configured to accommodate distributed energy resources," said Liang Min, LLNL associate program leader in energy infrastructure and HELICS co-project principal investigator. "Future grid will be an automated system with two-way flow of electricity and information between power plants and consumers—and all points in between. How we co-analyze the transmission-distribution infrastructure is very critical. The outcome from this project and associated projects will give utilities and the general public a way to analyze what the future grid will look like."

The GMLC was established to bring together leading experts, technologies and resources to collaborate on the goal of modernizing the nation’s grid and includes a dozen participating national laboratories.

Other collaborators on the GMLC HELICS project include Pacific Northwest National Laboratory and the National Renewable Energy Laboratory.