Project Group Virtualized Supercomputer

The Project Group Virtualized Supercomputer (Pg-Vsc) is engaged in virtualization of high performance computers (HPC). Our goal is to study how the benefits of virtualization can be used to compensate the performance loss, caused by hypervisor overhead. Furthermore we develop a sophisticated solution to manage several hypervisors in a common way.

Our target groups are companies, universities and research centers, which are interested in a persistent, scalable, flexible and easy to use technology to monitor and manage a heterogeneous hypervisor infrastructure. This innovative technology provides maximum flexibility and minimizes the performance losses by clever load balancing strategies and the administration overhead at the same time.

The end users are also developers of Resource Management Systems (RMS), who need a well arranged API to deal with virtual machines (VMs). This API is a great platform for research and testing of load balancing strategies and can be interesting for proofs of theoretical concepts.

In high performance computing (HPC), virtualization holds many possibilities to break the classical space sharing, where resources are statically assigned to Jobs. At this, the smallest resource unit is a node with possibly multiple CPU cores and a large amount of RAM. So if a non-parallel job that can only use one core and does not need much RAM space has to be executed, a waste of resources occurs. Due to virtualization, resources can be arbitrarily partitioned and assigned to a Job. Thus, such waste of resources can be avoided.
Furthermore, more resources can be saved by migrating jobs to more utilized nodes for a short period of time while these jobs do not use much CPU time (e.g. while doing I/O).
Another problem, which we solve, is the inflexibility of a cluster system. In many cases clusters run special operating systems adjusted to the clusters. This leads to recompilation of jobs to make them run on the cluster. By using virtualization every job obtains an isolated, customized environment in the form of a VM. The VM image, where this environment is installed can be created by customers themselves.

The Virtualized Supercomputer Project provides following features:

• Support of Xen, VMware, Hyper-V
• Well-arranged API to manage virtual machines
• Monitoring utility for VMs and hosts
• User Interface to deploy VMs
• Live migration of VMs
• Job scheduler with exchangeable scheduling algorithms
• Steering interface to VMs for application control and monitoring
• Authentification proxy within a HPC cluster

To learn more about Pg-Vsc please click here.