User mode, kernel mode and protection

Why it matters

Every SIGSEGV and every "Segmentation fault (core dumped)" is the CPU's protection hardware catching a program that stepped outside its box. The same hardware is why a crashed nginx worker does not take Postgres down, why a container cannot read the host's memory, and why the Meltdown mitigations of 2018 made system calls slower everywhere. Understanding the modes turns a cryptic crash into a precise statement about what the program tried and why the kernel refused.

The idea

Dual-mode operation means the CPU carries a mode bit saying whether the code running now is trusted. In kernel mode every instruction and every address is allowed. In user mode the CPU refuses a short list of instructions and any access to memory the page tables mark kernel-only. The CPU itself, not the kernel, enforces the difference on every instruction. The What an operating system does lesson described what the kernel owns; this is what makes the ownership real.

Rings and exception levels

x86 defines four protection rings, 0 to 3, but Linux and Windows use two: ring 0 for the kernel, ring 3 for programs. The current ring lives in the code segment register, and each page table entry has a bit saying whether ring 3 may touch the page. ARM calls the same idea exception levels: EL0 for applications, EL1 for the kernel, EL2 for a hypervisor, EL3 for firmware. The names differ; the model is the same.

Privileged instructions

Ring 0 reserves the instructions that would let a program escape its box: loading a page table base (mov cr3 on x86, msr ttbr0_el1 on ARM), disabling interrupts (cli), halting (hlt), port I/O (in, out), writing model-specific registers, switching modes. Everything else runs identically in both modes, which is why user code runs at full speed.

What happens on a violation

A privileged instruction in user code raises a general protection fault; touching a page it may not access raises a page fault marked as a protection violation. Either way the CPU switches to kernel mode, saves the faulting address and jumps to the handler the kernel registered at boot in the interrupt descriptor table (the vector table on ARM). For user code the answer is a signal: SIGSEGV for a bad access, SIGILL for a forbidden instruction; the process handles it or dies with a core dump. A fault in kernel code has no one to blame, so the kernel panics. Not every page fault is a violation: the TLB and page faults lesson covers the kernel bringing a page in and resuming the program.

Four ways into the kernel

Event	Who raises it	Synchronous?	Example
Interrupt	A device or timer	No, between any instructions	NIC signals a packet; timer tick
Exception	The CPU, on a failed instruction	Yes, at the instruction	Divide by zero, page fault
Trap	A deliberate instruction	Yes	Breakpoint (int3)
System call	The program	Yes	read, write, mmap

Textbooks group the synchronous ones as traps and the asynchronous ones as interrupts; what matters is who raised it and whether it was deliberate. The I/O and interrupts lesson follows a device interrupt to the process, and the System calls lesson covers the syscall path and strace, so neither is repeated here.

What a mode switch costs

A bare switch to ring 0 and back costs around 50 to 150 nanoseconds. The expensive part rides along: saving and restoring registers, and since Meltdown, Linux keeps kernel pages out of the user page tables (KPTI), so every entry and exit also swaps the page table base and may flush the TLB. That turned a 100-nanosecond crossing into several hundred, and syscall-heavy workloads lost 5 to 30 percent. A switch also pollutes caches and branch predictors, so its true cost is the cold misses afterwards, which is why the Context switching lesson measures a switch in microseconds.

Why hypervisors added a level

A guest kernel believes it runs in ring 0, but it must not control the real page tables or devices. Before 2006 hypervisors ran guests in ring 1 and rewrote their privileged instructions in software. Intel VT-x and AMD-V added VMX root mode and ARM added EL2: a level below the guest kernel to which the CPU exits whenever the guest does something the hypervisor must mediate. The Virtual machines and hypervisors lesson covers what exits cost.

Trade-offs

Protection is cheap while a program stays in its box and expensive each time it leaves. Every service becomes a crossing, and the post-2018 mitigations made each dearer, pushing designers towards batching, io_uring and user-space I/O. Rings protect only what page tables separate: two threads in one process can corrupt each other freely, and a kernel bug has nothing above it to catch it.

In an interview

Make the CPU the enforcer, not the kernel. "The CPU checks the mode bit on every privileged instruction and page access, so the kernel never inspects user code; it just installs the handler that runs when the hardware says no." On a crash: "A segfault is a page fault the kernel could not satisfy, so it sent SIGSEGV; the faulting address in the core dump says whether it was a null pointer, a freed buffer or a stack overflow."