Security threats and mitigations

Why it matters

Heartbleed read private keys out of OpenSSL's memory, Log4Shell turned a log line into remote code execution, and Dirty COW let any Linux user become root. Microsoft and Google both report that about 70 percent of their serious vulnerabilities are memory-safety bugs. The defences built in response shaped the systems you run, and in January 2018 the Meltdown patches made system-call-heavy services such as Postgres and Redis slower overnight.

The idea

A threat model starts with what an attacker wants: to run their code on your machine, to raise the rights of code already running, to read something they should not, or to make the service unavailable.

Threat classes

Malware is code the victim ran. Privilege escalation raises what running code may do, vertically (user to root) or horizontally (one tenant reading another's data). Memory-safety exploits turn a bug in C or C++ code, an overflow, a use-after-free, into control of the program. Denial of service exhausts a resource: fork bombs, SYN floods, a regular expression that backtracks for seconds. Side channels leak secrets through timing or cache state.

How a stack buffer overflow hijacks control flow

In C, char buf[64] lives in the current stack frame, and strcpy(buf, input) copies until it finds a zero byte. The stack grows toward lower addresses but writes into an array go upward, so 80 bytes of input run past the buffer, over the saved frame pointer and into the return address. When the function executes ret, the CPU pops that address and jumps to it. The 1996 original put machine code in the buffer and pointed the return address at it.

The mitigations that shaped modern systems

1. 1

   Stack canaries

   -fstack-protector puts a random word between the buffer and the return address and checks it before ret; a mismatch aborts with "stack smashing detected".
2. 2

   DEP, NX and W^X

   The MMU's NX bit makes stack and heap pages non-executable, so code written into a buffer cannot run; a page is writable or executable, never both (W^X). Attackers answered with return-oriented programming: chaining existing code fragments that end in ret.
3. 3

   ASLR and PIE

   Address space layout randomisation places stack, heap and libraries at random addresses each run, and a position-independent executable lets the program move too, so a ROP chain built on guessed addresses crashes.
4. 4

   Control-flow integrity

   CFI checks that returns and indirect calls go somewhere legitimate: a hardware shadow stack (Intel CET) keeps a second copy of return addresses the overflow cannot reach, and Clang's -fsanitize=cfi restricts indirect calls to functions of the right type.
5. 5

   Memory-safe languages

   Rust, Go and Java remove the bug class rather than catching the exploit; Android's share of severe memory-safety bugs fell from 76 percent in 2019 to 24 percent in 2024.

Meltdown, Spectre and the cost of KPTI

Speculative execution runs instructions before knowing whether they are needed and leaves footprints in the cache. Meltdown (2018) exploited Intel CPUs loading kernel memory speculatively before the permission check finished: a user process could read any kernel address by measuring which cache lines became warm. The fix is kernel page-table isolation (KPTI): the kernel keeps a second, nearly empty page table for user mode and switches CR3 on every entry and exit. That is why the trap in the System calls lesson and the switch in the Context switching lesson both became dearer: each crossing now flushes or re-tags TLB entries, and workloads making hundreds of thousands of small system calls per second lost 5 to 30 percent. Spectre mistrains branch predictors so a victim speculatively runs a gadget of its own code; retpolines and IBRS cost a few percent in branch-heavy code, and clouds stopped sharing a physical core between tenants.

Keeping the attack surface small

Most intrusions use a bug that already had a patch. Patch on a schedule (unattended-upgrades, kernel livepatch), build minimal images (distroless or scratch, so there is no shell to exec into), remove setuid binaries and unused services, and pin and scan dependencies. Log4Shell lived in a feature almost nobody used.

Trade-offs

Every mitigation is a tax on correct programs to raise the price of exploiting broken ones, and most are partial: ASLR falls to one leaked pointer, canaries guard only the stack, CFI narrows targets without removing them. Switching them off (mitigations=off) is defensible on a trusted single-tenant box and reckless on a shared host. None of this replaces least privilege and sandboxing: the goal is that an exploit which succeeds anyway lands in a process that can do very little.

In an interview

On a regression: "After the Meltdown patches our Redis nodes lost about 15 percent, because KPTI swaps page tables on every system call; pipelining recovered most of it by batching commands into fewer calls." Or: "The parser handles untrusted input, so it is in Rust; an overflow there would be remote code execution, and ASLR and canaries only make that harder, not impossible."