Swapping and thrashing

Why it matters

A server running out of memory rarely fails cleanly. First it slows down, sometimes by orders of magnitude, as the kernel shuffles pages between memory and disk; then something gets killed. Understanding page replacement, the page cache and thrashing explains why a database slows dramatically when its working set grows past RAM, why many server fleets run without swap, and how to read memory pressure before it becomes an outage.

The idea

Physical memory holds two broad kinds of pages: anonymous pages (heap and stack, with no file behind them) and file-backed pages (the page cache: file contents, executables, memory-mapped files). When free memory runs low, the kernel reclaims pages. Clean file-backed pages can simply be dropped, since they can be re-read from the file. Dirty file pages must be written back first. Anonymous pages have nowhere to go unless there is swap, a disk area where they can be written and later read back.

Choosing what to evict

The ideal policy evicts the page that will be used furthest in the future, which is impossible to know. Real kernels approximate least recently used: the hardware sets an accessed bit on use, and the kernel periodically scans and clears those bits. The clock (second-chance) algorithm sweeps pages in a circle, evicting the first whose accessed bit is clear and clearing the bits it passes. Linux keeps active and inactive lists (and newer multi-generational LRU) for anonymous and file pages, promoting pages that are used repeatedly and evicting from the inactive tail. This is the same family of ideas as cache eviction in the System Design track, applied to RAM.

Working sets and thrashing

A process's working set is the set of pages it actively uses over a short window. As long as the working sets of all running processes fit in RAM, page replacement only evicts cold pages and costs little. When they do not fit, every eviction removes a page that will be needed again soon, every access to it is a major fault, and the system spends its time moving pages between RAM and disk rather than computing: thrashing.

On a disk, a major fault costs milliseconds against a memory access of 100 nanoseconds, a ratio of tens of thousands. Even a small fraction of accesses faulting can multiply latency. NVMe narrows the gap but does not close it.

Swap on servers

Configuration	Behaviour under memory pressure	Fits
No swap	Anonymous memory cannot be evicted; once the page cache is squeezed, the OOM killer acts quickly	Latency-sensitive fleets that prefer a fast kill and restart over a slow death; many Kubernetes nodes
Small swap, low swappiness	Cold anonymous pages (startup code, idle data) can move out, freeing RAM for cache; heavy pressure still ends in OOM	General servers; gives the kernel room to manage memory well
Large swap	The system degrades gradually and may thrash for a long time before anything is killed	Desktops, batch machines where finishing slowly beats being killed
zram / zswap	Compressed swap in RAM; much faster than disk, trades CPU for capacity	Memory-constrained devices and some servers

Linux's swappiness setting biases reclaim between dropping file pages and swapping anonymous ones. Container platforms add cgroup memory limits, so pressure and OOM kills happen per container before the whole machine is affected.

Seeing pressure early

Free memory alone misleads, because Linux uses spare RAM for the page cache and reports it as used. Better signals: available memory (free plus easily reclaimable cache), the major page fault rate, swap-in and swap-out rates (vmstat's si and so columns), and pressure stall information (PSI): the share of time tasks are stalled waiting on memory, per system and per cgroup, which tools like systemd-oomd and Kubernetes can act on before thrashing sets in.

Trade-offs

Swap lets the kernel move genuinely cold pages out of RAM and absorb short spikes, and it can also turn an out-of-memory crash into hours of thrashing. No swap makes failure fast and visible and wastes RAM on cold pages. The page cache makes file I/O fast by using idle memory and competes with application memory under pressure. The right setting depends on whether you would rather be slow or be killed.

In an interview

Connect memory sizing to the working set: "The database's working set, the hot indexes and rows, must fit in its buffer pool plus page cache; past that, every query hits disk and latency multiplies. We alert on major fault rate and memory PSI, not just used memory, and run without swap on latency-critical nodes so a leak ends in a quick restart rather than a slow brownout."