Created:
1/25/2006 1:21:16 AM

Author:
Przemek Radzikowski

permalink [Permalink]





International Careers & Jobs - An international employment directory, reviewing world-wide top job sites




  • Home  ›
  • Articles  ›
  • Benchmarking VMware ESX Server 2.5 vs Microsoft Virtual Server 2005 Enterprise Edition



| More

Benchmarking VMware ESX Server 2.5 vs Microsoft Virtual Server 2005 Enterprise Edition

Deciding which virtualization environment to choose can be a frustratingly difficult task. I decided to push aside the price differences between the two products as well as all the media hype to put together a comprehensive comparison of ESX and Virtual Server.


 

 

Introduction

Deciding which virtualization environment to choose can be a frustratingly difficult task.  I decided to push aside the price differences between the two products as well as all the media hype to put together a comprehensive comparison of ESX and Virtual Server.

I had experimented with virtual environments in the past; VMware ESX, GSX and Workstation as well as Microsoft's VirtualPC and Virtual Server 2003.  With the release of Virtual Server 2005 I felt that a more scientific and methodical approach was needed to properly benchmark each environments performance.  These results would be used to decide which platform will be used as the virtualization host environment of choice.

Benchmarking Software

Hardware Specifications

Server Hewlett Pakard DL380 G4
Processor Dual Xeon 3.4MHz 800MHz FSB
Memory 4GB DDR2 Registered ECC in interleaved mode
Storage 4 x 36GB 10k RPM Hard Disk Drives
Storage Controller HP Smart Array Integrated 6i Controller



Virtualization Host Environment Software

Virtualization Guest System Setup

Operating system Microsoft Windows 2003 Server Enterprise Edition
Memory 1GB
Disk Drive 8GB fully expanded
Processors Combinations of 1 and 2
Hyper-threading Combinations of ON and OFF



Benchmarking Procedures

I started out with one physical processor installed in the server and Hyper-threading turned off.  After installing Windows Server 2003 Enterprise Edition I performed the standard set of tests using Sandra.  These results would become my benchmark against which all other results would be measured in terms of percentage gain or loss.

For tests involving Virtual Server 2005, it was installed on top of Windows Server 2003 Enterprise Edition.

The next step would be to shut down the BM (Bare Metal) server, turn on hyper-threading and run the tests again.  The next iteration would be to add a physical processor, turn off hyper-threading and re-run Sandra.  This exact procedure would be repeated until all possible CPU and hyper-threading combinations were tested.

ESX Server 2.5 adds another dimension into the combination equation with its support for 2 virtual CPUs.  If these results were going to be conclusive I would have to iterate through all possible combinations of Physical CPUs [C], Hyper-threading [HT] and Hosting Environment [BM] - Bare Metal; [VS] - Virtual Server; [ESX] - ESX Server 2.5.  These markings gave each combination of results a unique code which allows easy identification. 

Examples:

BM-W2k3-C2-HT0 = Bare Metal host with Windows 2003 Server as the operating system, 2 physical CPUs and Hyper-threading turned OFF.

ESX-W2k3-C22-HT1 = ESX Server 2.5 as hosting environment, with Windows 2003 Server as Guest operating system, 2 physical CPUs and 2 Virtual CPUs, Hyper-threading turned ON.

Results

Firstly I would like to clarify that any results in the following graphs are relative to the benchmark scenario or Bare Metal server with single Xeon processor.  CPU and Memory based benchmarks are expressed as a percentage (%).  The base system will always be 100% - combinations of hosting software and physical characteristic benchmark results are expressed as either a percentage (%) magnitude increase or decrease.  For example, a result of 33% indicates the tested configuration is operating at 33% capacity of the base system (which is at 100%). Any performance gain or loss is always relative to this base system - so we are always looking for results which are either close to the base system or better ( > 100%).  File IO and WinRAR testing is presented as a raw score.

Raw Test Results

The raw test results which were used to compile these graphs and infer conclusions may be viewed by clicking through to ESX-VS1.htm

Conclusion

It becomes clear with the numbers in front of us that Virtual Server is in serious trouble compared to ESX 2.5.  The main areas of concern are Memory Bandwidth and file system IO.  For a single processor system the Memory Bandwidth was running at 34% and 33% relative to a bare metal installation for non hyper-threaded and hyper-threaded respectively.  Similarly, the file system IO was at around 40% of the bare metal installation compared to ESX which hovered between 80% and 98% depending on the processor and hyper-threading combination.

Not surprisingly, Virtual Server had the worst WinRAR performance of the lot, performing the compression in 257 seconds compared to ESX of the same configuration with 178 seconds.  No matter what the combination of hyper-threading or physical processor Virtual Server had serious performance issues.  I decided to run the WinRAR test to see just how the Memory Bandwidth affected real life application performance.  In the test I compressed something which was at hand, at the time of testing I had a Windows ME installation CD handy so all WinRAR tests were performed using this 194MB CD image/folder.  All compression results are in seconds and show the time required to complete compression.

Recommendation

Looking at the Composite Test results graph it becomes quite easy to see that ESX 2.5 leads Virtual Server 2005 by a huge margin.  The most compelling results are both from ESX server; ESX-W2k3-C2-HT0 and ESX-W2k3-C22-HT1.  Both combinations of processor and HT achieve similar benchmark scores, however scrutinizing these closer I would be leaning towards ESX-W2k3-C22-HT1 purely on the higher Memory Bandwidth results.  ESX also supports virtual CPUs which if setup correctly perform almost as fast as a Bare Metal configuration.

In a real life scenario, if I had a server with a single physical CPU, given these results I would choose ESX-W2k3-C1-HT1, ESX Server 2.5, running a Windows 2003 Server guest operating system with a single virtual CPU and Hyper-threading turned on.

In the event that I had two physical CPUs, I would run using ESX-W2k3-C22-HT1.  Meaning my configuration would consist of ESX Server 2.5, running Windows 2003 Server guest operating system with two virtual CPUs and hyper-threading turned on.

Hope you find these results helpful in the rollout of your virtual infrastructure.

Acknowledgements

  • Anthony Sciorilli & HP for providing the test hardware.
  • Thierry Chabanne for initial hardware setup.

Further Reading

Read our earlier review: Benchmarking Microsoft Virtual Server 2005

Fujitsu-Siemens just released an interesting paper comparing VMware GSX and ESX Server, and Microsoft Virtual Server 2005 Enterprise Edition.

Since posting this article the Fujitsu-Siemens article has vanished.

 

permalink [Permalink] - Updated: Monday, October 28, 2013





| More

 

Articles of Interest


International Careers & Jobs - An international employment directory, reviewing world-wide top job sites


 
 
(c) Capitalhead Pty Ltd
Contact Capitalhead About Us Articles & Publications Partners Solutions & Services Products Valid XHTML Valid CSS