Virtual memory

Why it matters

Virtual memory is the reason two processes can both use address 0x400000 without colliding, why a bug in one program cannot overwrite another's data, why top shows a process with 20 GB of virtual size using 300 MB of real memory, why fork is cheap, and why the out-of-memory killer exists. Container memory limits, page cache behaviour and most "where did my memory go?" questions are answered here.

The idea

Programs never use physical memory addresses. Every address a program uses is virtual, and on every access the CPU's memory management unit (MMU) translates it to a physical address using tables the operating system maintains for that process. Each process gets its own tables, so each has its own private address space: on a 64-bit machine, a range far larger than the physical memory installed.

What the indirection buys

Benefit	How	Where you see it
Isolation	A process can only reach frames its own tables map; kernel pages are marked inaccessible from user mode	A crash or bad pointer stays inside one process
A simple, uniform layout	Every process sees the same address layout regardless of where its memory physically lives	Linkers and loaders can assume fixed regions
Sharing	Map the same physical frames into several processes	Shared libraries, the page cache, shared memory IPC
Copy-on-write	Share frames read-only and copy only when someone writes	Cheap fork; Redis background snapshots
Lazy allocation	Reserve address ranges now, allocate physical frames on first touch	malloc of 1 GB returns instantly
Files as memory	Map a file's contents into the address space; the kernel pages it in on access	mmap, how executables and databases load data
More memory than RAM	Move rarely used pages to disk and bring them back on demand	Swap, covered later in this module

Virtual size versus resident memory

Because allocation is lazy and mappings can be shared, three numbers describe a process's memory and they differ wildly. Virtual size (VSZ) counts every reserved range, including untouched heap, thread stacks and mapped files: often huge and mostly meaningless. Resident set size (RSS) counts physical pages currently in RAM for the process, including shared library pages that other processes also count. Proportional set size (PSS) divides shared pages among their users and is the fairest per-process figure. Container memory limits count resident memory (plus page cache charged to the container), which is the number to watch.

Overcommit and the OOM killer

Linux by default lets processes reserve more memory than RAM plus swap, betting that most reservations are never touched. When the bet fails and a page fault needs a frame with none free and nothing reclaimable, the kernel's out-of-memory killer picks a process (scored mostly by memory use) and kills it with SIGKILL. In a container, hitting the cgroup memory limit triggers the same mechanism scoped to the container, which is the usual meaning of exit code 137.

Where the translation lives

The tables are page tables, memory is managed in fixed-size pages (4 KB typically), and a small cache in the CPU, the TLB, keeps recent translations so most accesses do not walk the tables. The next lessons cover each of these, because they explain both the cost of virtual memory and its most common performance pathologies.

Trade-offs

Virtual memory buys isolation, sharing, lazy allocation and a uniform programming model, and costs translation hardware, page-table memory, TLB misses and page faults. Overcommit makes typical workloads more memory-efficient and replaces "allocation failed" errors with sudden OOM kills, which is a harder failure to handle. Almost every general-purpose system accepts these costs; some embedded and real-time systems do not.

In an interview

Use it to read memory numbers correctly and to explain kills: "The pod was OOM-killed at its 2 GB limit, exit code 137. Its virtual size is 9 GB, which is irrelevant; the resident set grew because the cache was unbounded. We bounded the cache and set the limit with headroom over measured peak RSS." That separates people who have debugged memory from people who have read top once.