Time series databases

Why it matters

Monitoring a fleet of ten thousand servers at one sample per second per metric produces billions of rows a day, nearly all of them appended in time order and queried as "average CPU for this service over the last hour". That shape is so regular that general-purpose databases waste most of their machinery on it. Time series databases exploit the regularity to ingest faster, compress twenty to one, and answer window queries in milliseconds, which is why observability, IoT and finance all run on them.

The idea

A time series is a sequence of (timestamp, value) points identified by a set of tags: cpu_usage{host=web-7, region=eu}. A time series database stores many such series with three assumptions baked in: writes are appends in roughly time order, queries filter by tags and aggregate over time ranges, and old data loses detail value and can be summarised or dropped.

Storage shaped by time

Data is partitioned by time into chunks (an hour, a day), so a query for the last hour opens one or two chunks and retention is deleting whole old chunks rather than running expensive deletes. Within a chunk, values for one series are stored together and compressed with columnar encodings that exploit regularity: delta-of-delta for timestamps that tick every second, XOR or Gorilla encoding for slowly changing floats, run-length for repeated values. Compression ratios of 10 to 30 times are routine, which is what makes retaining months of high-resolution data affordable.

Downsampling and retention

Nobody needs per-second CPU from eight months ago. Downsampling (continuous aggregates, rollups) pre-computes coarser series, five-minute and one-hour averages, minimums and maximums, as data ages, and retention policies drop raw data after days and rollups after months or years. Dashboards automatically read from the resolution appropriate to the window. This tiering is the difference between a store that costs a fortune and one that costs almost nothing per series.

The cardinality problem

Each unique combination of tag values is a separate series with its own index entry and storage. Tagging metrics with a user id, a request id or a container id that changes on every deploy creates millions of short-lived series, bloats the index, and slows every query: this is high cardinality, and it is the way time series systems die. Rules: tags are for dimensions you will group by with bounded value sets (service, region, status code); unbounded identifiers go in logs or traces, not metric tags. Monitor the active series count as a first-class metric.

Queries and the ecosystem

Queries are window aggregates: rate of a counter over five minutes, p99 of a histogram, group by region, compare to last week. Purpose-built languages (PromQL, Flux) and SQL extensions (TimescaleDB's time_bucket, ClickHouse functions) make these concise. Alerting evaluates such queries continuously. Common choices: Prometheus for infrastructure metrics with a pull model and local storage, with Thanos, Cortex or Mimir for long-term and multi-cluster; TimescaleDB when you want it inside Postgres; InfluxDB; ClickHouse when the workload shades into analytics over events.

	Time series store	Relational table with a timestamp
Ingest	Millions of points per second, append-optimised	Tens of thousands of rows per second, index maintenance per row
Compression	10 to 30× with time-aware encodings	Modest, generic
Window queries	Native, pre-aggregated via rollups	Possible, slow at scale without partitioning
Retention	Drop whole chunks	Expensive DELETE or manual partition management
Fits	Metrics, sensors, prices, events keyed by time	Business data that happens to have a timestamp