CFS and modern schedulers

Why it matters

Nearly every server you deploy runs the Linux scheduler, and most containers you run are limited by its cgroup controls. The single most common container performance surprise, a service with low average CPU that still suffers terrible tail latency because it is throttled, comes straight from how the scheduler enforces CPU quotas. Understanding fair scheduling, weights and quotas lets you set requests and limits that do what you intend.

The idea

Linux's Completely Fair Scheduler (CFS, the default from 2007, refined into EEVDF in 2023) does away with fixed time slices and queues of urgency. Its goal is to approximate an ideal processor that runs every runnable task simultaneously at an equal share of speed.

Virtual runtime

Each task carries a virtual runtime: how much CPU time it has received, scaled by its weight. The scheduler always runs the task with the smallest virtual runtime, the one that has received the least of its fair share so far. As a task runs, its virtual runtime grows; once it is no longer the smallest, another task gets the core. A task that sleeps does not accumulate virtual runtime, so when it wakes it is near the front, which is how interactive tasks get quick service without any explicit "interactive" classification.

Weights and nice values

Each nice value maps to a weight; one nice step changes the CPU share by roughly 10 percent relative to a competitor, so nice 0 versus nice 5 is roughly a 3 to 1 split. Virtual runtime advances more slowly for heavy-weighted tasks, so they run longer before yielding. Fairness is also applied between groups: with cgroups, CPU is shared fairly between containers or users first, then among the tasks inside each one, so a container with 100 threads cannot crowd out one with 2.

EEVDF: fairness plus latency

Pure virtual-runtime fairness gives every task its share over time but says nothing about how soon a latency-sensitive task runs. EEVDF (earliest eligible virtual deadline first), now Linux's default, gives each task a virtual deadline based on its requested slice: tasks that ask for short slices get earlier deadlines and run sooner, while long-running tasks still get their fair share. It replaces a pile of heuristics in CFS with a cleaner rule and better latency behaviour.

Multicore reality

1. 1

   Per-core run queues

   Each core has its own queue and tree, avoiding a global lock that would not scale.
2. 2

   Load balancing

   Periodically, and when a core goes idle, the scheduler migrates tasks from busy cores to idle ones, preferring nearby cores (same cache, same socket, same NUMA node) to keep caches warm.
3. 3

   Affinity

   Tasks tend to stay on the core where their cache is warm. You can pin them explicitly (taskset, cpusets) for latency-critical work or isolate cores from general scheduling entirely.
4. 4

   Wake-up placement

   When a task wakes, the scheduler picks a core: an idle one nearby if possible, otherwise the one it last ran on.

CPU limits in containers

Kubernetes CPU requests become cgroup weights: under contention a pod gets CPU in proportion to its request. CPU limits become a quota per period (by default, the pod may use limit times 100 ms of CPU in each 100 ms window). A multi-threaded service can burn its whole quota in the first few milliseconds of a window with many threads running in parallel, and then is throttled, unable to run at all, until the window resets. Average CPU looks low, yet requests that arrive during throttling wait up to tens of milliseconds, wrecking tail latency.

Mitigations: set limits well above typical bursts or omit them for latency-sensitive services (relying on requests for fairness), size thread pools to the limit rather than the host's core count, and watch the cgroup throttling counters.

Trade-offs

Fair scheduling gives every task its share and good interactivity without classification, at the cost of the per-task bookkeeping and occasional latency issues EEVDF addresses. Per-core queues scale but need load balancing that can move tasks away from warm caches. Quotas enforce hard isolation at the cost of throttling stalls that averages hide.

In an interview

The container point lands well: "The service showed 30 percent average CPU and terrible p99. Throttling counters showed it hitting its CPU limit within each 100 ms period because 32 runtime threads ran in parallel against a 2-core limit. We sized the runtime to 2 cores and raised the limit, and p99 dropped by an order of magnitude."