Inodes and directories

Why it matters

Inodes explain a cluster of practical behaviours: why a disk can be "full" with gigabytes free, why renaming a file is the standard way to update it safely, why a hard link and a symlink behave differently when the target moves, and why storing millions of small files in one directory slows everything down. They are also the reason file names are not part of a file's identity.

The idea

On Unix-like file systems, each file is represented by an inode (index node), a fixed-size record identified by a number. A directory is itself a file whose content is a list of entries, each mapping a name to an inode number. The name lives in the directory; everything else about the file lives in the inode.

What an inode holds

Field	Notes
Type and permissions	Regular file, directory, symlink, device; read, write, execute bits for owner, group, others
Owner and group	Numeric user and group IDs
Size	In bytes; may exceed allocated blocks for sparse files
Timestamps	Modified (mtime), changed metadata (ctime), accessed (atime, often relaxed for performance), sometimes creation time
Link count	How many directory entries point here
Data location	Block pointers (classic ext2/3) or extents: runs of contiguous blocks (ext4, XFS)

Classic inodes held a dozen direct block pointers plus indirect blocks (pointers to blocks of pointers) for larger files. Modern file systems use extents, "1,000 blocks starting at block 800", which describe large contiguous files in a handful of entries and make sequential I/O efficient.

Links

A hard link is simply another directory entry pointing to the same inode; the inode's link count goes up, and the data survives until every link is removed and every open descriptor is closed. Hard links cannot cross file systems or point to directories. A symbolic link is a separate small file whose content is a path; it can point anywhere, including across file systems, and breaks if the target is moved or deleted. Deployment tools often switch versions by repointing a symlink (current → releases/42).

Atomic rename

rename within one file system atomically replaces one directory entry with another: readers see either the old file or the new one, never a half-written mix. That gives the standard safe-update pattern: write the new contents to a temporary file in the same directory, fsync it, rename it over the old name, and fsync the directory so the rename itself is durable. Configuration tools, package managers, editors and databases all rely on it.

Inode limits and directory size

Many file systems allocate a fixed number of inodes when created (ext4 by default, roughly one per 16 KB of space). Millions of tiny files (session files, cache shards, mail spools) can use every inode while blocks remain free, so writes fail with "No space left on device" and df -i reveals it. XFS and btrfs allocate inodes dynamically and avoid this.

Directories with millions of entries were historically slow because lookups scanned a list. ext4 and XFS index large directories with hashed B-trees, so lookup is fast, but listing, backing up, deleting and syncing such directories is still painful, and many tools load the whole listing. The usual fix is fanning files out into a hashed hierarchy (ab/cd/abcdef…), which object stores do implicitly with key prefixes.

Caches in front of them

The kernel caches inodes and directory entries (the dentry cache) so repeated path lookups and stat calls avoid disk. Workloads that stat or open millions of distinct files churn these caches, which is one reason scanning huge trees is slow even on fast storage.

Trade-offs

Separating names from inodes enables hard links, atomic rename and open-but-deleted files, at the cost of extra lookups per path. Fixed inode tables are simple and can run out; dynamic allocation avoids that with more complex structures. Extents make large files efficient; huge numbers of tiny files remain expensive in metadata whatever the layout.

In an interview

Use atomic rename for safe updates: "The config agent writes the new file to a temp path in the same directory, fsyncs it, renames it over the old one and fsyncs the directory, so a crash leaves either the old or the new file, never a torn one." And for storage design: "Millions of small objects go to an object store, or are packed into larger files, not one file per object in a single directory."