LinuxHPC.org/Cluster Builder 1.3
Condor Cycle Scavenger
Condor Cycle Scavenger
By LinuxHPC.org and Cluster Resources
| Condor Cycle Scavenger |
|
Condor Cycle Scavenger is a software framework for coarse-grained distributed parallelization of computationally intensive tasks. It can be used to manage workload on a dedicated cluster of computers, and/or to farm out work to idle desktop computers—so-called cycle scavenging.
Integration
Condor runs on Linux, Unix, Mac OS X and contemporary Windows operating systems. Condor can seamlessly integrate both dedicated resources (rack-mounted clusters) and non-dedicated desktop machines (cycle scavenging) into one compute environment. Many hundreds (probably thousands) of organizations use Condor to manage workload, and to leverage added value from their estates of desktop PCs and workstations.
In the world of parallel jobs, Condor supports the standard MPI and PVM (Goux, et al. 2000) in addition to its own Master Worker "MW" library for extremely parallel tasks.
Features
Condor can run both sequential and parallel jobs. Sequential jobs can be run in several different "universes", including "vanilla" which provides the ability to run most "batch ready" programs, and "standard universe" in which the target application is re-linked with the Condor I/O library which provides for remote job I/O and job checkpointing. Condor also provides a "local universe" which allows jobs to run on the "submit host".
Other Condor features include "DAGMan" which provides a mechanism to describe job dependencies, and the ability to use Condor as the front-end to submit jobs to other distributed computing systems (such as Globus). The Condor Project is an active participant in the grid computing field.
Status
Condor is developed by the Condor team at the University of Wisconsin-Madison and is freely available for use. While Condor follows an open source philosophy (it's licensed under a BSD-like license), the source code is only available on a case-by-case basis.
By way of example, the NASA Advanced Supercomputing facility (NAS) Condor pool consists of approximately 350 SGI and Sun workstations purchased and used for software development, visualization, email, document preparation, etc. Each workstation runs a daemon that watches user I/O and CPU load. When a workstation has been idle for two hours, a job from the batch queue is assigned to the workstation and will run until the daemon detects a keystroke, mouse motion, or high non-Condor CPU usage. At that point, the job will be removed from the workstation and placed back on the batch queue.
All text used in this article is available under the GNU Free Documentation License. It uses material from the Wikipedia article "Condor cycle scavenger".