Homework: VMM Benchmarking
Create a VM and install a Linux guest operating system on it.
You have to measure the
performance of your VM on the following benchmarks.
- emptyloop : A compute-intensive user process
- forkbomb : Forks one million processes, each of which exits gracefully
- getpid : A user level process repeatedly calls the getpid() system call
- cp : Copy a large file (say 100MB or more) within the same directory
- find : Use "find . -name " on the root directory
- iscp : Copy a file from a remote host to the local host.
- oscp : Copy a file from the local host to a remote host.
- lincompile : Compile the Linux kernel from source.
- apachebench: Run Apache Server on the local host. Request 4KB static pages using 4 concurrent ApacheBench threads.
- memcached: Run memcached on the local server. Use the
memslap benchmark, part of the libmemcached client library, to load
the server and measure it's performance. Memslap should run on a remote host and
send a random sequence of memcached get (90%) and set (10%) requests to the server
and measure the request completion rate. The memslapd should make 64 concurrent
requests from 4 threads (with other parameters set to their default values).
- ping : The ping latency between the host and the guest.
We will provide you with two physical machines with Ubuntu Linux installed
on them and connected through Gigabit ethernet. You should use these machines
for benchmarking. Because these machines have to be shared among many students, we
will assign time-slots to each student to perform his/her benchmarking experiments.
You must fully test your code on your local machines before running them on the test
infrastructure, to avoid wasting time.
You should present a bar chart of this type: sample graph.
You should report performance relative to that on the native host.
For each experiment, perform 10 trials and plot the minimum time among those
10 trials. Report performance
for VMware and KVM. Choose different virtual devices and benchmark again.
For example, use e1000, ne2000, etc. on KVM for network card. Similarly try different
network devices on VMware. Similarly, use different storage adapters (IDE, SCSI)
and different virtual disk formats (raw, qcow2, etc.) and report performance
for each.
For VMware, test performance using the two different virtualization
modes, namely binary-translation (software-only) and hardware-assisted.
Note that only 32-bit guests work with binary translation.
We highly recommend that you use Makefiles and automated scripts to automate
your result generation. This will make it very easy for you to obtain results
for other variants, once have written scripts to obtain results for one variant.
We provide you with some sample scripts and Makefile here:
- benchmark.c: A C program which takes as arguments the number of trials and the names of the benchmarks and returns their time of completion. Note that this C file uses the
rdtsc
instruction on x86 to measure time. rdtsc counts the number
of CPU cycles. For machines using DVFS (Dynamic Voltage and Frequency
Scaling), the number of cycles may not directly correspond to time. Hence,
it may be better to use other ways to measure time (e.g.,
gettimeofday).
- Other useful scripts (you may or may not use them): hw1_files
- A nice tool for generating bar graphs: http://www.burningcutlery.com/derek/bargraph/
Use qemu-img (available with Qemu distribution) to convert
disk image between formats raw, qcow2,
vmdk, etc. Also, experiment with different network
configurations (e.g., tunnelled, NAT, bridged, etc.).
Clarifications
- Issue with forkbomb
- DVFS and rdtsc
- QEMU Network configuration for Apache/memcached
- SCSI Disks
Submission Instructions
Neatly present your findings (performance data) using appropriate
graphs and tables in a report. Also include brief explanations on
why you think the performance for different variants is the way
it is. Email your report to Aashish (your TA). You can
find Ashish's contact on the Administrivia
page.