VMware/VMmark

VMmark
VMware VMmark - VMmark Virtualization Benchmarks

Beta Details
[http://blogs.vmware.com/performance/2010/07/vmmark-20-beta-overview.html VMware: VROOM! Blog: VMmark 2.0 Beta Overview]:
 * "As I mentioned in my last blog, we have been developing VMmark 2.0, a next-generation multi-host virtualization benchmark that models not only application performance in a virtualized environment but also the effects of common virtual infrastructure operations. This is a natural progression from single-host virtualization benchmarks like VMmark 1.x and SPECvirt_sc2010. Benchmarks measuring single-host performance, while valuable, do not adequately capture the complexity inherent in modern virtualized datacenters. With that in mind, we set out to construct a meaningfully stressful virtualization benchmark with the following properties:"


 * Multi-host to model realistic datacenter deployments
 * Virtualization infrastructure workloads to more accurately capture overall platform performance
 * Heavier workloads than VMmark 1.x to reflect heavier customer usage patterns enabled by the increased capabilities of the virtualization and hardware layers
 * Multi-tier workloads driving both VM-to-VM and external network traffic
 * Workload burstiness to insure robust performance under variable high loads

VMware introduces VMmark 2.0 beta | Virtualization - InfoWorld:
 * "VMware's new benchmark will focus on the entire virtualized data center rather than a single host server like the new virtualization benchmark from SPEC
 * If Mark Twain were alive today, he might have popularized yet another saying for those of us in the computer industry: "Lies, damned lies, and benchmarks."

Benchmark results are often met with skepticism and controversy because of concerns that the results can be manipulated to make one vendor or platform appear better than another."

VMware introduces VMmark 2.0 beta | Virtualization - InfoWorld:
 * "VMware has finally released the first public beta of VMmark, the company's virtualization benchmarking tool. First announced just before VMworld 2006 (the company's annual virtualization conference for developers, partners, and end-users), the latest VMmark released to the public has undergone some key changes since that original specification was developed in order to simplify benchmark setup and execution."

Enroll for the beta, here. (Ken Note: unable to find download)

VMmark 1.x
Note: VMmark® 1.x is now permanently retired. It will remain available for download for academic purposes only. The VMmark 1.x results page will remain available for reference.

Requirements

 * One server with 2 CPUs, 6GB RAM, 80G disk space and a 1Gb NIC
 * One client PC for every tile, each with 2 CPUs, 2GB RAM and 15G available disk space
 * A private network for connecting clients to the server

Workload Software
 * Operating systems: Microsoft Windows Server 2003 Release 2 Enterprise Edition (32-bit and 64-bit); SUSE Linux Enterprise Server 10 (32-bit and 64-bit)
 * Applications: Microsoft Exchange 2003; SPECjbb2005; SPECweb2005; MySQL; dbench

Purchased Software:
 * Microsoft Outlook 2003 (standalone or included in Microsoft Office 2003)
 * Microsoft Windows Server 2003 Release 2 Enterprise Edition (32-bit)—three 32-bit copies per tile (two for virtual machines and one for that tile’s client system), and one 64-bit copy per tile (for the Java server virtual machine)
 * Microsoft Exchange Server 2003 Enterprise Edition (This is available as part of the Microsoft Developer’s Network (MSDN) Universal subscription.)
 * SPECjbb2005 Benchmark
 * SPECweb2005 Benchmark

dbench
See VMmark dbench

VMmark 1.1 Download
http://downloads.vmware.com/d/info/datacenter_downloads/vmware_vmmark/1_x

VMmark 1.1 Results
VMmark Results


 * http://www.vmware.com/files/pdf/vmmark/VMmark-Cisco-2010-04-06-UCS-B250-M2.pdf
 * http://www.ciscosystems.ac/en/US/prod/collateral/ps10265/cisco_UCS_C460m1_VMmark.pdf
 * http://www.vmware.com/files/pdf/vmmark/VMmark-HP-2009-06-02-dl385g6.pdf