Thomas Zacharia
News-Sentinel photo by Michael Patrick

Thomas Zacharia, ORNL's associate director for computational sciences, stands inside a high-performance data storage system. It has the capacity to store the information equivalent to 25 Libraries of Congress.

from Knoxville News-Sentinel
October 21, 2002
original URL:,1406,KNS_4257_1484545,00.html

Calculating the future

Super machines vital to test ideas ranging from climate to biology

By Frank Munger, News-Sentinel senior writer
October 21, 2002

In January 1995, with the arrival of the Intel Paragon XP/S 150, Oak Ridge National Laboratory boasted the world's fastest supercomputer. It was capable of 150 billion calculations per second.

By the time the next world rankings were posted four months later, the Intel machine already had dropped to No. 3 on the list, and four years after that the once-prized supercomputer was put on the junk heap: literally sold for scrap.

Such is the turbocharged evolution of computers.

The federal research lab is hustling to keep pace because science in the 21st century depends on high-performance computing.

"It's critical to everything we do," said Bill Madia, the ORNL director.

Many believe that computing now constitutes the third leg of science, joining the theoretical and experimental realms.

"We use these enormous supercomputers to develop models based on theories and mathematics and fundamental first principles," said Buddy Bland, a group leader in ORNL's Center for Computational Sciences. "It allows us to simulate experiments that previously we would never be able to do."

Bob Ward, a former lab administrator who heads the Computer Sciences Department at the University of Tennessee, noted, "On many things we're getting to the limits of physical experiments. You have to have that computer capability for large science applications."

No component of research is moving as quickly as high-end computing or information technology in general, according to Thomas Zacharia, ORNL's associate director for computational sciences.

"What's more is these computers are accelerating scientific advancements in other fields. Computing is central to nanotechnology and biotechnology. It's central to materials and climate-change predictions and fusion calculations," Zacharia said. "We are now approaching a level of computing capability where it can begin to query things in a substantive way that we could not before - with atomic-scale resolution."

ORNL's reigning supercomputer is an IBM Power 4 - nicknamed Cheetah - that reached full operation earlier this year. At 42 trillion calculations per second, Cheetah's computing capability ranks among the top 10 in the world.

Also in the lab's working stable of supercomputers are Eagle (an IBM SP) and Falcon (Compaq SC 64-node).

Those three collectively provide 6 teraflops, 6 trillion calculations per second, of computing power and make Oak Ridge one of the top computer research centers in the United States, rivaling the best defense laboratories.

"If you're not in the game, you're at a strategic disadvantage," Bland said.

But Japan seriously upped the ante earlier this year when it unveiled the Earth Simulator, a stunning machine with a capability of 40 teraflops -surpassing the collective computing power of the rest of the world's top 10 machines.

Oak Ridge National Laboratory has been chosen to develop a U.S. response to the Earth Simulator, and in mid-August ORNL signed a cooperative agreement with Cray Corp. to develop the world's fastest supercomputer.

Within the next few months ORNL will acquire a Cray X1 supercomputer system and begin working on new ways to stretch its capabilities in massively parallel processing to new limits.

The Japanese reportedly spent $1 billion developing the Earth Simulator. The United States plans to surpass it within 18 months to two years, perhaps spending half as much.

"This laboratory will get there ... and will in a reasonably short amount of time be the world leader in scientific computation," Ray Orbach, DOE's science chief, said during a late-summer visit to Oak Ridge.

Lab officials emphasize that computational excellence is much more than a ranking on the number-crunching scale. It's about people.

After a late-summer phone conversation with Tetsuya Sato, director of Japan's Earth Simulator laboratory, Zacharia acknowledged that the international competition for computer supremacy will be difficult to win.

"Dr. Sato is going around the world rounding up the best scientists. The question that we all have to ask ourselves is, 'Do we want our best scientists working for the Japanese or should they be working for Oak Ridge or another lab here?' "

The biggest hurdle, however, may be getting the necessary money from Congress, he said.

ORNL has a notable computer heritage, acquiring one of the earliest machines in 1950 for research on nuclear-powered aircraft.

Lab engineers also spurred development of the Oak Ridge Automatic Computer and Logical Engine, or ORACLE, which became operational in 1954 and was - for a time - the fastest computer in the world. It reportedly could do about 100 man-years of calculations in an eight-hour work shift.

But today's lofty position in the computing hierarchy is remarkable considering ORNL was a non-factor just a decade ago. In 1992, an Oak Ridge team shocked the computer establishment by winning a national competition under the Federal High Performance Computer and Communications Initiative.

"Nobody expected it," Bland said. "Oak Ridge National Laboratory was not on anybody's radar screen."

As a result of that achievement, the Department of Energy sanctioned Oak Ridge as one of two sites, the other being Los Alamos National Lab in New Mexico, for advanced computing research centers. A decade later, that victory looms even more important to the lab's well-being.

Ward and Ed Oliver, an ex-ORNL executive who's now with DOE's Office of Science in Washington, are credited with putting together the proposal that landed a world-class machine and established Oak Ridge as a supercomputing center.

According to Ward, the laboratory proposed using the new supercomputer to simulate groundwater movement for environmental programs, work on advanced materials and do high-level nuclear physics. The first two ideas were approved and funded.

What separated Oak Ridge's approach from those submitted by other labs was a promise to involve scientific experts from around the nation, including top universities. Other national labs were relying largely on their in-house staff to do the research projects.

Ward fondly remembers the effort, and he's proud he played a role in building the program's reputation. But the UT official said the turnaround at ORNL occurred nearly a decade earlier, in 1985, when then-Director Herman Postma decided to establish a supercomputing capability.

Instead of trying to compete with other labs for the best technologies that existed at the time, Postma chose to focus on the future and jump a generation ahead. As a result, ORNL was able to acquire Serial No. 1 of Intel's first commercial parallel computer, an IPSC/860.

"It was in the same ballpark as a supercomputer, and it was powerful enough to do some serious work," Ward said.

Years later, Oak Ridge continues to pursue an individual strategy in supercomputing.

Unlike other top computing centers, ORNL limits the uses and users for its top-of-the-line supercomputers, and thus gets more bang on its research projects - especially the task of modeling the world's changing climate.

Lawrence Berkeley National Laboratory in California, for instance, accommodates about 2,000 users a year at its computer research center. Oak Ridge is equipped with similar capabilities but limits its user list to about 200.

Zacharia said the top machines are reserved primarily for research in biology, climate change and advanced materials, and that approach won't change.

"If anything, we're going to be even more focused on tackling really important challenges," he said. "We don't want to be like a honeybee going from one flower to another. We want to stay the course until we have an impact and then go on to another field."

ORNL has had an association with most of the major computer makers.

The laboratory, for instance, recently has been working with IBM on a project known as Blue Gene, as well as having two IBM machines in the supercomputing center.

Zacharia notes that Oak Ridge worked years ago with Cray, then acquired a big machine from Intel and then IBM and then Compaq and now is preparing to receive the latest from Cray once again.

"Our task," he said, "is not to be the private-sector company building machines or selling machines. Our task is doing science, and assembling the best computers to do the science. That's my job."

ORNL is not interested in exclusive arrangements with any company, he said.

Zacharia predicts the lab will have a petascale computing capability - 1,000 trillion calculations per second - before the end of the decade. That's hundreds of times greater than what's now available.

"We will get there. If not the leader of the pack, we will be in the lead pack," ORNL's computing chief said. "I have to pinch myself about what I just said. It's not that far away. It keeps me awake at night."

Copyright 2002, KnoxNews. All Rights Reserved.
Mirrored with permission.

This page is part of the CSMD web archive and is not maintained.
Please visit for the latest CSMD information.