Archived News Releases are no longer maintained. Click here for recent News Releases.

  Contact: Don Johnston
  Phone: (925) 423-4902
  E-mail: johnston19@llnl.gov
  FOR IMMEDIATE RELEASE
November 7, 2005
NR-05-11-04

New visualization cluster installed on world’s fastest supercomputer

LIVERMORE, Calif. — The National Nuclear Security Administration’s (NNSA) program to ensure the safety, security and reliability of the nation’s nuclear deterrent in the absence of testing now has one of the world’s most powerful visualization computers installed at Lawrence Livermore National Laboratory (LLNL).

VisIt Visualization on Powerwall
A powerwall in Livermore's new Terascale Simulation Facility shows a simulation of results from an experiment mixing two liquids of different densities. Powerwalls work by aggregating, or "tiling," the separate images from many projectors to create one seamless image.

Named “gauss,” the new visualization cluster was built by GraphStream Incorporated, in partnership with NNSA’s Advanced Simulation and Computing (ASC) Program, to support the visualization and data analysis needs of LLNL’s BlueGene/L (BG/L), which is the fastest computer in the world. BG/L’s recent breakthrough calculations relate to materials aging and turbulence, providing new scientific insights needed for NNSA stockpile stewardship.

Steve Louis, LLNL Subprogram Lead for ASC Computational Systems and Software Environment, said, “We realized that our current visualization solutions would be inadequate to handle the massive data that would be generated by BG/L codes. At early meetings, researchers commented that BG/L would produce output so massive it could only be handled by BG/L itself. Gauss provides a solution.”

Delivered in September, the gauss cluster is a 256-node dual-CPU multi-teraFLOP/s cluster supercomputer based on Tyan™ S2895 motherboards with AMD Opteron™ CPUs running at 2.4GHz. In addition to the two CPUs and 12 GB of memory, each node contains a high-end NVIDIA™ Quadro 4500 graphics card, a feature that differentiates gauss from traditional computing clusters. According to Dale Southard, LLNL’s hardware architect for gauss, “If you look at an individual node of gauss, its design is very similar to an ultra-high-end consumer gaming machine – fast CPUs, lots of RAM, and a video card that gamers would die for.”

Rayleigh-Taylor Instability Simulation
This image of a Rayleigh-Taylor instability was produced on gauss using LLNL's VisIt software. The Rayleigh-Taylor instability occurs when a light fluid is accelerated into a heavy fluid, and is a fundamental fluid-mixing mechanism that is important to inertial confinement fusion applications. The simulation was produced from a full-machine 64K node run on BlueGene/L using Miranda code. It represents more than 10 million CPU hours (more than 1,000 years worth of calculation on a single modern PC).

View animation in Quicktime format
View in Windows Media format

The Quadro 4500 graphics cards are leading-edge products, similar to consumer model 7800GTX cards, but have twice the memory and provide better performance for advanced OpenGL™ operations like pixel readback. The 3D engine in each Quadro 4500 card can perform more than 150 billion floating point calculations per second. By harnessing these cards in parallel, gauss can visualize extremely large datasets containing billions of triangles in real time. Because gauss is based on Linux, it can run modern software tools such as LLNL’s VisIt, Kitware’s ParaView™, and CEI’s EnSight™, distributing the visualization work across multiple video cards. For other software tools, open-source Chromium cluster rendering middleware provides a way to distribute the rendering load of unmodified OpenGL applications.

The gauss nodes are connected by a Voltaire™ InfiniBand network. This interconnect provides transfer rates of more than 800 megabytes per second to individual nodes. Notably, gauss uses OpenIB™ software drivers to run the InfiniBand network – one of the first clusters of this size to do so, and the first large InfiniBand cluster of any kind at LLNL. According to Southard, “LLNL was able to run MPI jobs that performed more than 1.8 trillion operations per second on gauss within two weeks of receiving the hardware. This rapid deployment and stability is a tribute to the maturity of InfiniBand hardware and Linux OpenIB drivers.”

Along with new technologies, gauss also builds on existing software expertise. Like other clusters at LLNL, gauss runs LLNL's CHAOS (Clustered High Availability Operating System), based on Red Hat Enterprise Linux. CHAOS provides seamless administration across the entire range of LLNL Linux clusters, and requires no modification of node hardware or firmware. In addition to InfiniBand for intra-cluster communication, gauss shares a 900 TB Lustre™ filesystem with BG/L over 256 separate gigabit ethernet links that can move data in parallel, providing high-bandwidth, zero-copy access to simulation data. The ability to leverage existing software and tools on the newest commodity hardware provides quick, cost-effective solutions to critical NNSA computational problems such as visualization of massive BG/L data.

Founded in 1952, Lawrence Livermore National Laboratory has a mission to ensure national security and apply science and technology to the important issues of our time. Lawrence Livermore National Laboratory is managed by the University of California for the U.S. Department of Energy's National Nuclear Security Administration.



Founded in 1952, Lawrence Livermore National Laboratory is a national security laboratory that develops science and engineering technology and provides innovative solutions to our nation's most important challenges. Lawrence Livermore National Laboratory is managed by Lawrence Livermore National Security, LLC for the U.S. Department of Energy's National Nuclear Security Administration.