IT Editorial

Traditional internal data centers are grappling with record-high energy prices, fueling the demand for efficiency within an organization’s infrastructure.  According to statistics from the Uptime Institute, a 451 Group company, the average data center is 10 percent utilized, and therefore 90 percent idle.  Additionally, only about 3 percent of a data center’s electricity consumption is spent crunching data.  The remainder is spent on idleness, redundancy and other energy overhead.  

Prior to the advent of virtualization and the cloud, this type of inefficiency was a requisite for decoupling applications from each other and enabling reliability through redundancy.  Critical applications had a production and disaster-recovery environment, both fully provisioned in separate locations to ensure that uptime could be guaranteed even with outages and failure.  In contrast, the cloud enables dynamic access to resources as a service that can expand and contract an application’s use of servers and storage in multiple data centers based upon need.  Users of internal and public cloud methodologies have the promise of building efficient systems because of this flexibility.

The cloud model, where applications utilize only the virtual machines and storage needed to run, enables three forms of efficiency:

• Managing the Peaks: Organizations no longer have to purchase and power infrastructure to handle their peak needs for an application, only to have it sit idle.
• Economies-of-Scale Efficiencies:  By provisioning large-scale server capacity, used by multiple applications only when it is needed, and charging only for the resources an application uses, cloud helps drive costs down and make deployment more cost efficient.
• Increased Reliability, Lower Costs: Properly handling disaster-recovery scenarios requires provisioning of multiple data centers and networking contracts, which increases costs and lowers efficiency.  Cloud reduces this by removing complexity and enabling access to multiple compute and storage sources at large economies of scale for increased reliability and lower costs.
  

For internal clouds, built on server farms and managed by companies like VMware, virtualization has enabled these efficiencies.  The advent of virtualization makes leveraging infrastructure easier.  Application stacks can be secured and isolated from one another while running on the same, otherwise-idle infrastructure – opening up many possibilities for increasing efficiency in a cloud model. 

High throughput and performance computing (HTC/HPC) is a good place to look for demand for these “idle”  compute cycles in a data center.  Specifically, driving up utilization to save energy and money is the core mission of an open source project called Condor.  Condor is a fully featured, high-throughput batch scheduler created by the University of Wisconsin- Madison.   

Condor can help save energy by enabling calculation portability using virtual machines to run server applications on other infrastructure when those machines are idle.  Using green-computing policies, users can also hibernate machines to save money when there are no calculations to complete.  Virtualization makes it possible to run calculations on servers that otherwise don’t have the right operating system or software installed.  Once those calculations are complete, machines can be hibernated, to save additional energy until they are needed the following morning.

Most companies perform calculations in the form of risk analysis, product design and simulations, bioinformatics, financial analysis, computational fluid dynamics and Monte Carlo simulation, among many others.  Combining this latent demand with virtualization and Condor’s ability to aggregate computation, organizations can save significant amounts on energy and new equipment, while performing calculations that lower risk and speed time to market.  

Purdue University is an example of these kinds of efficiency gains.  Purdue’s DiaGrid project uses Condor to run across more than 28,000 processors, harvesting computation cycles to achieve an unheard of 1-2 percent idle time, as opposed to the 90 percent mentioned earlier.  Scientists currently run calculations for genomic, scientific and financial research on DiaGrid.  Purdue estimates it has saved more than $3 million and 2,000 square feet of datacenter space by merely reusing idle compute it already owns.   

In this age of virtualization and application portability, organizations can use cloud computing to acquire external compute resources when needed, provision for median rather than peak needs and harvest compute cycles from idle resources they already have.  Using modern software and cloud methodologies, organizations can have idle rates much lower than the data center average of 90 percent.  As cloud adoption continues to grow, more data centers will begin to lower energy usage and save money.

By Jason Stowe, CEO, Cycle Computing