Making the best of your virtual environment

There is no such thing as an IT department with too much money. That’s always been true, but budgets right now are universally tight. For many organizations, the question is not which “nice-to-have” optimization to leave off the list this year, but which essential project to delay.

Shoestring budgets are not the only driving force behind an increasing number of organizations looking to virtualize their IT environments. Besides speed, reliability and flexibility, making the best possible use of scarce resources is a big factor.

According to IDC, companies in Singapore that virtualize their servers are expected to save US$8 billion in costs by 2020.Overall, the Asia-Pacific region will see US$98 billion worth of cost savings until 2020 in the areas of server spending, power and cooling, floor space and cost of manpower and overheads.

Evidently, virtualization has introduced significant benefits to enterprises and environmental sustainability. The savings of virtualization come from the fact that the more virtual servers a company can run on a single physical machine, the more energy it will save and the less often it has to buy new hardware. In most organizations, the business case for virtualization is built on the assumption of a “consolidation ratio” of between 10 and 20 to 1– in other words, every physical machine will host 10-20 virtual machines (VMs).

The sprawl

Once virtualization has been implemented, users quickly notice many other advantages. One of the key benefits is the sheer convenience of creating new virtual machines. Instead of going through a lengthy procurement process to buy a new server, a development or project team can commission as many new VMs as required in minutes.

However if housekeeping isn’t done properly, the proliferation of VMs can drastically slow down the environment and fatally undermine the business case for going the virtual route in the first place. It’s such a common problem it even has a name: VM sprawl.

Part of the reason VM sprawl happens is the ancient human instinct to keep anything useful “just in case”. So even though the project is completed, all the files are archived and there are full backups of all the VMs which can be restored at a moment’s notice – people still hesitate to delete the VMs themselves. Often the machine is un-registered so it’s easy to forget about, but it remains in a kind of zombie state.

Superficially it seems less risky to let old VMs hang around in case they are needed – after all, hardly anybody ever gets in trouble for NOT deleting something. However in reality, it’s VM sprawl that poses the real risks.

Zombie VMs

The first risk is the one that really kills a business case: Every zombie VM is still using up valuable system resources, especially expensive storage. Let’s say it’s been allocated 500Gb of data – even if that storage space is empty, it still can’t be used because it’s reserved for that VM. It’s like putting a traffic cone in an empty parking space: It instantly turns free space into wasted space.

Combine zombie VMs with other junk data like old ISO files and defunct system snapshots and you can push an entire storage array to breaking point surprisingly quickly. The larger the organization, the more people there are contributing to the sprawl and the faster trouble will happen, no matter how well-resourced it is.

There’s also a compliance risk: How many operating system licenses does an organization have? Every zombie VM is using up one of them. When the vendor comes to do an audit and finds you over the limit, “whoops we seem to have overlooked that one” is not going to be a good enough excuse.

Business processes that prevent wanton creation of new VMs are one way to solve the problem – but a process that overcomplicates things also undermines the benefit of flexibility virtualization brings in the first place.

Thin provisioning needs to be implemented, particularly for SAN storage space which is notoriously expensive. Thin provisioning means configuring a VM with all the storage required – but actual storage space will only be released when it’s needed, up to the maximum it’s been configured for.  With this arrangement, the most efficient use of available storage pool is ensured.

Out with the old

Another solution is to be rigourous about clearing out old and unused files once they’ve been located. This is not a job that can be done manually as it needs specialized tools that can not only identify junk files but also delete them safely.

Another problem that undermines the business case for virtualizing is misallocation and over-allocation of system resources.

The easiest way to create a new VM is to do it from a template. For example, every new machine gets, two CPUs, 8Gb of RAM and 100Gb of storage space. That’s a good, average specification however, not every VM has average needs. Some will need more resources, and some will need much less.

Assigning too many resources is wasteful and inefficient. It needlessly ties up resources that could be utilized elsewhere. On the contrary, assigning too few resources hurts the performance.

Finding the right balance between efficiency and performance needs the right tools – meaning tools specifically designed for managing virtual environments.

Virtualization works because it allows organizations to over-commit resources, knowing that most processes don’t need as much as they are allocated. It’s like the way airlines overbook flights, knowing that there’s a fairly predictable proportion of people who won’t turn up. It is because of this and the multiple layers of abstraction created in the process of virtualizing, that it is almost impossible to understand the true resource usage of any VM using the same tools and approaches we use in physical environments.

Get the tools right

The right monitoring and reporting tools are needed to ensure that virtual over commitment doesn’t become real. These tools will also allow companies to manage VMs dynamically, allocating and taking away resources accordingly.

With the rules of virtual environments being so different, it’s important to choose monitoring tools that give recommendations, not just reports. What you need is not just one more report, but an expert system that contains a lot of the specialist knowledge any organization running a virtual environment needs.

Apart from tools, a system that can identify problems affecting specific servers, departments or applications is a must have. Throwing more RAM at a slow machine, for example isn’t always the right solution. What if the real problem is disk related, or an application with a memory leak? Monitoring tools need to supply that information – in the specific context of a virtualized environment with overcommitted resources.

Human decision making is always paramount, because humans know things about context and the future unlike machines. If the monitoring tool notices that between January and June a particular server was under-utilized, it may recommend switching resources away from that machine. The human who knows that the company’s financial year-end is coming up in July will also know not to implement that particular recommendation.

Between an IT manager’s knowledge of the business context, and a good monitoring tool’s knowledge of the virtual environment, it is possible to run a virtualized IT shop that delivers both better performance and lower cost, meeting the ROI targets set by the business.

There is no doubt that organizations will increasingly continue to virtualize their IT infrastructure. Traditional IT policies, legacy management and reporting tools are fast becoming inefficient in meeting the demands of an environment delivering increased performance, scale and availability. A rethink of IT management to meet these demands and ensure businesses in Singapore reap the most from virtualized infrastructure is a no brainer. This will eliminate the potential pitfalls that could seriously undermine the business case for virtualizing.

Charles Clarke is Technical Director for APAC, Veeam Software