Purdue’s West Lafayette campus and a growing number of partner campuses around Indiana are maximizing their investment in computer hardware and putting what otherwise would be unused, wasted compute cycles to work on a variety of research and educational projects.

The partnership is showing how higher education institutions can join together in a proven, mutually supportive way to provide significant computational resources using machines the partners already have purchased and free software, said John Campbell, Purdue associate vice president for information technology.

To do that, Purdue employs Condor, a University of Wisconsin-developed workload management system for doing compute-intensive jobs with pooled networked machines—on desktops, in labs, server arrays and high performance computing clusters—when they are idle or underutilized, such as at night, on weekends, during lunch, or when their owners are just busy doing something else. Purdue manages one of the largest Condor pools in the world, located both on and off its West Lafayette campus.

Named DiaGrid, the pool is centered at Purdue in West Lafayette and now includes machines at Indiana University, Indiana State University, the University of Notre Dame, Purdue’s Calumet and North Central campuses, and Indiana University-Purdue University Fort Wayne. The grid is designed to incorporate computers at other locations across the state and, indeed, the nation as new members join. At the end of 2008, it boasted more than 20,000 processors, and growing.

The resource has been used by researchers at Purdue and elsewhere for purposes such as imaging the structure of viruses at near-atomic resolutions; simulating the early stages of the Solar System’s formation; projecting the reliability of Indiana’s electrical supply; modeling the spread of water pollutants; and identifying millions of potential new forms of zeolites, silicate minerals widely used to catalyze chemical reactions on an industrial scale.  Purdue’s Condor pool provides computational resources to researchers on both the Open Science Grid and the National Science Foundation’s TeraGrid.

The system also has been used to help create a virtual version of a pharmacy clean room for training student pharmacists; to render a fly-through animation of a proposed satellite city that could serve as a refuge for Istanbul, Turkey, in the event of a catastrophic earthquake; and to animate scenes for Nano Factor, a game designed to interest junior high students in science and engineering.

Condor—and by extension DiaGrid—is designed for general-purpose, high-throughput distributed computing across administrative boundaries and without requiring a shared file system. Capable of delivering large amounts of computational power over a long period of time, secure and yet relatively easy to use, it’s excellent for things like parameter sweeps, Monte Carlo simulations, or nearly any serial application, from encoding video to pattern searches of a genome.
Basically, the system automatically handles the parceling of jobs to processors in the pool, keeps track of progress, migrates work to another node when an owner needs a machine being used and notifies users when tasks are completed. This makes Condor “opportunistic” in tapping unused computational resources wherever they’re available, all with very little user intervention. An account isn’t even required on the remote machines.

Nonetheless, machine owners have extensive control, through configuration files, over what jobs execute and under what conditions, including the ability to define when a machine is idle (identified by low load and no keyboard activity, for example), or allowing jobs only at certain times and on certain days.

DiaGrid is led  by Information Technology at Purdue (ITaP), the central information technology organization at Purdue’s West Lafayette campus, through the Rosen Center for Advanced Computing, ITaP’s research and discovery arm. To pool computational resources spread around the state, the grid takes advantage of I-Light, the high-speed fiber-optic state network connecting Indiana campuses to each other, the Internet and national research networks such as the Internet2 and National LambdaRail.

Interested in collaborating? Please contact: