Virtual machines and hypervisors

Why it matters

Almost every cloud server you rent is a virtual machine, and most containers run inside one. Understanding the hypervisor explains cloud-specific behaviour you otherwise cannot: CPU "steal" time, why instance types with dedicated cores cost more, why network and disk performance depend on instance size, why serverless platforms can start an isolated VM in a fraction of a second, and why multi-tenant isolation relies on VMs rather than containers alone.

The idea

A virtual machine (VM) is a software-defined computer: virtual CPUs, memory, disks and network cards on which a complete, unmodified guest operating system runs. A hypervisor (virtual machine monitor) creates VMs, gives each a slice of the real hardware, and keeps them isolated from each other.

How guests run at near-native speed

A guest kernel expects to control the CPU, page tables and devices, which a hypervisor cannot allow. Early hypervisors rewrote sensitive instructions or trapped them in software. Modern CPUs provide hardware-assisted virtualization (Intel VT-x, AMD-V): a guest mode in which the guest kernel runs its own privileged code directly, and only specific events (some instructions, interrupts, access to unmapped device memory) cause a VM exit to the hypervisor. Nested page tables (EPT, NPT) let the hardware translate guest-virtual to guest-physical to host-physical addresses itself, so memory access needs no hypervisor involvement, at the price of longer page-table walks on TLB misses.

Devices

Emulating a real network card or disk controller instruction by instruction is slow. Paravirtual devices (virtio) are designed for virtualization: the guest's driver talks to the hypervisor through shared-memory queues, batching requests. Clouds go further and offload networking and storage to dedicated hardware cards (AWS Nitro), and SR-IOV lets a physical NIC present virtual functions that a VM uses directly, bypassing the hypervisor on the data path.

What virtualization costs

Cost	Cause	Where you see it
CPU overhead	VM exits for privileged operations and interrupts	A few percent on modern hardware; more for syscall- and interrupt-heavy workloads
Memory translation	Two-level page walks on TLB misses	Large, random-access heaps; mitigated by huge pages
Steal time	The hypervisor runs other guests on the physical core your vCPU wanted	The st column in top and vmstat; common on shared and burstable instances
Noisy neighbours	Other tenants share caches, memory bandwidth, NICs and disks	Variable latency on shared instances; dedicated hosts remove it
I/O limits	Virtual devices are rate-limited per instance	Network and disk throughput tied to instance size

A vCPU on many clouds is a hyperthread, not a full core, so 8 vCPUs may be 4 physical cores. Burstable instance types accumulate CPU credits and are throttled to a baseline once they are spent, which looks like sudden steal or slowness.

VMs versus containers, and microVMs

A VM virtualizes hardware and runs its own kernel, so a compromised guest kernel still faces the hypervisor boundary: the strongest isolation available short of separate machines. Containers, next lesson, share the host kernel and are lighter but isolate less. MicroVMs combine them: Firecracker (behind AWS Lambda and Fargate) runs a minimal device model and boots a stripped kernel in about a hundred milliseconds with a few megabytes of overhead, giving each function or container its own kernel. gVisor and Kata Containers pursue similar goals.

Trade-offs

VMs buy strong isolation, full OS flexibility and hardware consolidation at the cost of some CPU and memory overhead, slower boot and duplicated kernels. Shared tenancy is cheap and unpredictable; dedicated hosts and cores are predictable and expensive. MicroVMs narrow the gap between VM isolation and container speed.

In an interview

Explain cloud behaviour from the hypervisor: "The p99 spikes lined up with steal time on burstable instances, which had exhausted their CPU credits; moving to fixed-performance instances removed them. For running untrusted customer code we use Firecracker microVMs, because a shared kernel is not a strong enough boundary for multi-tenant execution."