Containers: namespaces and cgroups

Why it matters

Containers are how most software is deployed, and most of their surprises come from forgetting what they are: processes on a shared kernel. That one fact explains why a container starts in milliseconds, why it sees the host's CPU count unless the runtime says otherwise, why exceeding a memory limit means an OOM kill rather than swapping, why containers on one host share kernel vulnerabilities, and why "root in a container" is a security conversation.

The idea

A container is one or more ordinary Linux processes started with two kernel features applied: namespaces, which give them their own view of system resources, and cgroups (control groups), which limit and account for the resources they use. Add a root file system from an image (next lesson) and a few security restrictions, and you have what Docker or Kubernetes calls a container. There is no guest kernel; a process in a container is visible in the host's process list with a host PID.

Namespaces: what a container can see

Kubernetes puts all containers of a pod in one network namespace, which is why they reach each other on localhost and cannot both listen on the same port.

Cgroups: what a container can use

Cgroups (version 2 on current systems) organise processes into a tree and apply controllers to each node:

1. 1

   CPU

   Weights for proportional sharing under contention (Kubernetes requests) and quota per period as a hard cap (limits). Bursty multi-threaded services hit the quota early in each period and are throttled, as the scheduling module showed.
2. 2

   Memory

   A hard limit on anonymous memory plus page cache charged to the group. When reclaim cannot get the group under its limit, the OOM killer kills a process in it, exit code 137. memory.high adds a softer threshold that throttles allocation first.
3. 3

   I/O

   Bandwidth and IOPS limits and weights per device.
4. 4

   PIDs

   A cap on process count, which stops a fork bomb in one container from exhausting the host.

Cgroups also provide accounting: per-container CPU, memory and I/O usage is what monitoring agents and kubectl top read.

What is not isolated

The kernel is shared. A kernel vulnerability or a container escape affects every container on the host. Some resources are not namespaced at all: the system clock, kernel modules, most of /proc/sys, and kernel keyrings. Programs that read host-wide information get host-wide answers: nproc and older runtimes see every core rather than the CPU limit, and free inside a container shows host memory, which is why runtimes must be told the container's limits (modern JVMs and Go versions read the cgroup). Resources such as the page cache, memory bandwidth and L3 cache are shared, so neighbours can still interfere.

Security layers on top

Runtimes add defences because namespaces alone are not a security boundary: dropping Linux capabilities (root inside lacks most privileges), seccomp filters on system calls, AppArmor or SELinux profiles, read-only root file systems, running as a non-root user, and user namespaces so container root is unprivileged on the host. For untrusted multi-tenant code, platforms add a VM boundary (Firecracker, Kata, gVisor), as the previous lesson described.

Trade-offs

Containers buy fast startup, density and a consistent packaging format by sharing the kernel, which costs isolation strength compared with VMs. Hard limits protect neighbours and turn overuse into throttling and OOM kills that averages hide. Running with fewer privileges is safer and occasionally breaks software that assumed it was root.

In an interview

State what the boundary is: "Each service runs in a container with CPU and memory limits from load tests, as a non-root user with dropped capabilities and the default seccomp profile. For customer-supplied code we add a microVM per job, because containers share the kernel." When diagnosing: "Exit code 137 with the container at its memory limit is the cgroup OOM killer; the runtime must know the limit or it sizes itself for the host."