Service discovery

Why it matters

In a monolith, calling another module is a function call. In a fleet of services, calling another service means knowing which of its instances exist right now, which are healthy, and where they are, when autoscaling and deploys change the answer continuously. Hard-coded addresses break on the first deploy; stale caches send traffic to dead instances. Service discovery is the plumbing that makes "call the orders service" work, and every platform provides a version of it that you should be able to explain.

The idea

A service registry holds the current set of instances for each service name with their addresses, ports, health and metadata. Instances register on start and deregister on stop (or are removed when health checks fail). Callers resolve a service name to healthy instances and pick one, with load balancing. The design choices are who does the lookup and balancing, and how the registry learns about health.

Client-side versus server-side

	Client-side discovery	Server-side discovery
Who resolves	The caller queries the registry and picks an instance	The caller sends to a stable address (load balancer, proxy, virtual IP); the router resolves and forwards
Hops	One; direct connection	Two; through the router
Load balancing	In the client library; can be smart (latency-aware, zone-aware)	In the router; one place to configure
Coupling	Every language needs a discovery-aware client	Clients are plain; the platform does the work
Examples	Netflix Eureka with Ribbon, gRPC with xDS, Consul-aware clients	Kubernetes Services, AWS load balancers, Envoy or NGINX in front, service mesh sidecars

Server-side discovery has largely won because it keeps application code simple and centralises policy, with the mesh sidecar as the current form: discovery happens in a proxy beside each instance, giving client-side efficiency with server-side simplicity.

Health and freshness

A registry is only useful if it reflects reality. Instances send heartbeats or the registry probes a health endpoint (the Heartbeats lesson's trade-offs apply); unhealthy instances are removed from answers within a few seconds. Registration should be automatic on startup and deregistration on graceful shutdown, with a drain period so in-flight requests finish before an instance disappears. Separate readiness (can serve traffic) from liveness (process is alive): a starting instance is alive but not ready, and should not receive traffic yet.

DNS as discovery and its trap

The simplest discovery is a DNS name that resolves to the current instances (A records or SRV records with ports). It needs no client library and works everywhere. The trap is caching: resolvers, operating systems and language runtimes cache answers, sometimes ignoring TTLs (the JVM once cached forever by default), so callers keep sending to instances that were terminated minutes ago. Use short TTLs, make sure every layer honours them, and prefer a platform that updates DNS and load balancers together.

The platforms

Kubernetes gives every Service a stable virtual IP and DNS name; kube-proxy or the CNI routes to ready pods, and readiness probes gate membership, so discovery is invisible to the application. Consul combines a registry, health checks, DNS and an API, and can drive a mesh. etcd and ZooKeeper are the consistent stores many registries are built on. Cloud load balancers with target groups and auto-registration are server-side discovery for VM and container fleets. Whichever platform, the application should depend only on a name.

Operating it

Make discovery itself highly available (a replicated registry) and make clients degrade gracefully when it is unreachable (keep the last known good list). Add metadata to registrations (version, zone, canary flag) so routing can prefer same-zone instances or send a percentage to a canary. Watch for registration leaks (instances that never deregister) and for flapping (instances marked unhealthy and healthy repeatedly), which usually indicates an overly aggressive health check.

Trade-offs

Client-side discovery buys a hop and smart balancing at the cost of per-language libraries and policy spread across clients. Server-side buys simple clients and central policy at the cost of a hop and another component to run. DNS-based discovery is universal and simple and suffers from caching. The platform usually makes the choice for you; the design responsibility is to depend on names, register and deregister correctly, and gate on readiness.

In an interview

Keep it short and correct: "Services call each other by name. On Kubernetes that resolves to a Service with readiness-gated endpoints, so autoscaling and deploys are invisible to callers; instances drain before termination. If we add a mesh, the sidecar does discovery and balancing with zone preference." If the platform is VMs: "Consul registry with health checks, short-TTL DNS, and clients that keep the last known instance list if the registry is unreachable."