Originally appeared in November 28, 2003 HPC Wire

ORNL To Design High-Speed Experimental Network

Big science requires big computers that generate vast amounts of data that must be shared efficiently, so the Department of Energy's Office of Science has awarded Oak Ridge National Laboratory $4.5 million to design a network up to the task. Raymond L. Orbach, director of DOE's Office of Science, said, "Advanced computation and high-performance networks play a critical role in the science of the 21st century because they bring the most sophisticated scientific facilities and the power of high-performance computers literally to the researcher's desktop. Both supercomputing and high-performance networks are critical elements in the department's 20-year facilities plan that Secretary of Energy Spencer Abraham announced Nov. 10."

The prototype dedicated high-speed network, called the Science UltraNet, will enable the development of networks that support high-performance computing and other large facilities at DOE and universities. The Science UltraNet will fulfill a critical need because collaborative large-scale projects typical today make it essential for scientists to transfer large amounts of data quickly. With today's networks, that is impossible because they do not have adequate capacity, are shared by many users who compete for limited bandwidth, and are based on software and protocols that were not designed for petascale data.

"For example, with today's networks, data generated by the terascale supernova initiative in two days would take two years to transfer to collaborators at Florida Atlantic University," said Nageswara Rao of ORNL's Computer Science and Mathematics Division.

Obviously, Rao said, that isn't acceptable, and he, Bill Wing and Tom Dunigan of ORNL's Computer Science and Mathematics Division are heading the three-year project that could revolutionize the business of transferring large amounts of data. Equally important, the new UltraNet will allow for remote computational steering, distributed collaborative visualization and remote instrument control. Remote computational steering allows scientists to control and guide computations being run on supercomputers from their offices.

"These requirements place different types of demands on the network and make this task far more challenging than if we were designing a system solely for the purpose of transferring data," Rao said. "Thus, the data transmittal requirement plus the control requirements will demand quantum leaps in the functionality of current network infrastructure as well as networking technologies."

A number of disciplines, including high-energy physics, climate modeling, nanotechnology, fusion energy, astrophysics and genomics will benefit from the UltraNet.

ORNL's task is to take advantage of current optical networking technologies to build the prototype network infrastructure that enables development and testing of the scheduling and signaling technologies that will be needed to process requests from users and optimize the system. The UltraNet will operate at 10 gigabits to 40 gigabits per second, which is about 200,000 to 800,000 times faster than the fastest dial-up connection of 56,000 bits per second.

The network will support the research and development of ultra-high-speed network technologies, high-performance components optimized for very large- scale scientific undertakings. Researchers will develop, test and optimize networking components and eventually make them part of Science UltraNet.

"We're not trying to develop a new Internet," Rao said. "We're developing a high-speed network that uses routers and switches somewhat akin to phone companies to provide dedicated connections to accelerate scientific discoveries. In this case, however, the people using the network will be scientists who generate or use data or guide calculations remotely."

The plan is to set up a testbed network from ORNL to Atlanta, Chicago and Sunnyville, Calif.

"Eventually, UltraNet could become a special-purpose network that connects DOE laboratories and collaborating universities and institutions around the country," Rao said. "And this will provide them with dedicated on-demand access to data. This has been the subject of DOE workshops and the dream of researchers for many years."