Ports and sockets

Why it matters

"Address already in use", "too many open files", "cannot assign requested address" and connections that hang at high load are all socket-level problems, and they hit services that are otherwise healthy. Sockets are also the API every networked program sits on, so knowing what they are explains how one server handles a hundred thousand connections on a single port, how load balancers and proxies hold connections open, and which limits you will hit first.

The idea

A port is a 16-bit number (0 to 65,535) that lets one IP address serve many programs. A socket is the operating system object a program uses to send and receive on the network; to the program it is a file descriptor. A TCP connection is identified by its four-tuple: source IP, source port, destination IP, destination port. As long as any one of the four differs, connections are distinct, which is why a server listening on port 443 can hold vast numbers of connections at once: each has a different client address or client port.

The server side

1. 1

   socket and bind

   The server creates a socket and binds it to an address and port, such as 0.0.0.0:443 (all interfaces) or 127.0.0.1:8080 (local only). Ports below 1024 traditionally need elevated privileges.
2. 2

   listen

   The socket becomes a listening socket with a backlog: a queue of connections whose handshakes have completed and that are waiting for the application.
3. 3

   accept

   Each accept call takes one completed connection off the queue and returns a new socket for it. The listening socket keeps listening; the new socket carries that one conversation.
4. 4

   read and write

   The application reads and writes on the connected socket; the kernel handles TCP underneath, buffering in per-socket send and receive buffers.

The client side and ephemeral ports

A client usually does not choose its port; the kernel assigns an ephemeral port from a range (on Linux typically 32,768 to 60,999, about 28,000 ports). Since the four-tuple must be unique, a client can hold at most that many simultaneous connections to one destination IP and port, minus those still in TIME_WAIT. A proxy or service making very many short connections to one backend hits "cannot assign requested address" when the range runs out: widen the range, spread across more destination IPs, and above all reuse connections.

The limits you will hit

Socket options worth knowing

SO_REUSEADDR lets a restarted server bind its port even while old connections sit in TIME_WAIT, avoiding "address already in use" after a restart. SO_REUSEPORT lets several processes or threads each have their own listening socket on the same port, with the kernel spreading new connections across them, which removes a single accept queue as a bottleneck. TCP_NODELAY disables Nagle's algorithm, which otherwise delays small writes to batch them; latency-sensitive request-response protocols usually set it. SO_KEEPALIVE and its timing settings send probes on idle connections so dead peers are detected and NAT or load balancer mappings stay alive.

Sockets beyond TCP

UDP sockets have no listen or accept: one socket receives datagrams from anyone. Unix domain sockets use a file path instead of an IP and port and carry traffic between processes on one machine without the network stack, which is faster and is how many databases, sidecars and container runtimes talk locally.

Trade-offs

Sockets give programs a uniform, file-like interface to the network while the kernel does the protocol work, at the cost of per-socket kernel memory and system-call overhead that matter at very high connection counts or packet rates. Defaults are tuned for modest workloads; busy servers need raised descriptor limits, backlogs and buffers, and clients that make many connections need reuse rather than a bigger port range.

In an interview

Show you know where the ceilings are: "Each gateway instance holds 200,000 client connections, so file descriptor limits are raised to a million, the accept backlog to 4,096, and we use SO_REUSEPORT so each worker has its own accept queue. Upstream, the gateway keeps pooled keep-alive connections to services so it never runs out of ephemeral ports."