Vivian Voss

What Isolation Actually Costs

containers docker freebsd benchmarks

The Bench Marker ■ Measured 25 August 2026

On 15 March 2013, at PyCon in Santa Clara, Solomon Hykes stepped on stage for a five-minute lightning talk titled "The Future of Linux Containers". The demo became Docker, the company followed the demo, and the industry followed the company. By any commercial measure it was the most successful five minutes in the history of packaging software.

At the moment the applause started, the isolation being announced as the future had been shipping in FreeBSD's base system for thirteen years. This column measures the difference between those two sentences.

The problem that needed a product

Docker exists because Linux is fragmented. The platform never agreed on a package format: deb, rpm, apk, pacman's tarballs. Beneath that sit two C libraries with different opinions, and beneath those a long argument about init systems. A binary built on one distribution carries no promise of running on the next, and by 2013 that gap was costing real money on real mornings.

Docker's answer was capitulation dressed as engineering: if the distributions cannot agree on a system, ship the whole system with every application. The fragmentation stayed exactly where it was, shrink-wrapped into every parcel, and the industry traded distribution fragmentation for base image fragmentation, which is the same argument one floor up: ubuntu or alpine, slim or distroless, whose libc, whose patch schedule.

To say the fair thing once and plainly: as a mechanism for moving an artefact from a build server onto a Linux host, this is honest engineering, and in 2013 it beat everything else on offer. The question for a bench is different. What does the arrangement cost, measured against a system that never had the political problem in the first place?

Two roads to one web server you docker CLI dockerd root · 144 MB idle containerd registry 100 pulls / 6 h image: 7 layers · 63.3 MB 151 packages inside pkg mirror no account · no meter nginx + pcre2 10 MB · 1.02 s one web server, serving Every station on the upper road is a cost that did not exist before the tool arrived.

The scale, tared

The legend requires isolation to have been heavy before 2013, so the first measurement tares the scale. The bench is a rented machine in Falkenstein: AMD Ryzen 5 3600, 64 GB, NVMe, FreeBSD 15.0-RELEASE. The vessel is a jail, rolled back to a golden ZFS snapshot before every series. The probe is fifty lines of C that write four kilobytes to a file and read them back, two hundred thousand times over; a second variant adds cap_rights_limit() and cap_enter(), locking the process into Capsicum capability mode, where it may keep its work but may never open anything new.

capbench, 200,000 x 4 KiB write+read, medians of five runs
plain process, host          460.9 ms
plain process, jail          462.1 ms
capability mode, jail        457.8 ms
jail start, cold             0.17-0.18 s

The capability-mode median came in nine tenths of a per cent below the unrestricted one, which is the polite way of saying the two are indistinguishable. Host and jail sit within a millisecond and a half of each other across half a second of work, and a jail cold-starts in under a fifth of a second. Twenty-six years in the base system, and the price of the fence is too small for this bench to find.

On this kernel there was never anything heavy to make light. The lightning talk was solving a different operating system's problem.

The parcel, opened

nginx, ordered twice, both sides counted as compressed bytes over the wire so that neither side is flattered. The Docker side was weighed without Docker, through a registry token and the manifest API: nginx:latest for amd64 arrives as 63,327,223 bytes in seven layers; nginx:alpine brings it down to 26,253,895 bytes, and there the published comparisons usually stop. This one kept going. Unpacked in the same jail and read from the image's own dpkg ledger, nginx:latest carries 151 Debian packages and 324 executables in its system paths, in the service of exactly one web server. Every package chosen by the image maintainer rather than by you, every one ageing on a schedule you do not control, every one attack surface that arrived uninvited. That is the fragmentation of 2013, vacuum-packed and posted to every recipient, worldwide, daily.

Bytes over the wire, same web server nginx:latest 63.3 MB nginx:alpine 26.3 MB pkg install nginx 10.1 MB Compressed transfer bytes, measured 25 August 2026; registry weighed via manifest API, no Docker involved.

The FreeBSD side of the same errand: pkg install nginx fetches the web server at 1,715,152 bytes and exactly one dependency, pcre2, at 8,411,385 bytes. Ten megabytes over the wire in two parcels, 1.02 seconds from command to installed, a service start in two hundredths of a second, eleven megabytes resident.

The structural point sits underneath the numbers. FreeBSD has distributions of its own: GhostBSD, TrueNAS, pfSense, OPNsense. Yet no FreeBSD Docker exists, and none is missed, because base system and packages are separate by design, one kernel, one userland, one source tree, and a package built for the release runs on all of them. Nothing in that design ever needed rescuing.

The workaround's own economy

Every line in this section describes a cost that did not exist before the workaround arrived, and each is quoted from the industry's own paper trail.

The daemon: dockerd and containerd reside in memory before the first container earns its keep, around 144 megabytes idle by a published Debian measurement (our own rig for that figure is not finished, so it is borrowed and marked as such in the protocol below). The pair runs as root, and with live-restore off by default, restarting it takes every running container down. The registry: a metered counter in front of every install, one hundred anonymous pulls per six hours, the announced cut to ten per hour merely postponed; toomanyrequests is standard operating procedure on a busy CI day. The licence: Docker Desktop at 9 to 24 dollars per seat per month once a company crosses 250 employees or ten million in revenue. The cadence: every CVE among those 151 packages means rebuilding the image and reshipping 63 megabytes to the whole fleet, on the maintainer's schedule, while the base system answers the same CVE with a 1.7-megabyte package. The debugging: "works in Docker" has quietly replaced "works on my machine" as the sentence that opens a lost afternoon.

And between the nodes runs the low-grade kerfuffle the industry has learnt to file under normal operations. We have not measured it and say so plainly; the operators have documented it for us, in an entire curated archive of public Kubernetes post-mortems whose recurring characters are conntrack tables, DNS timeouts and overlay networks, and in the CNCF's own survey, where a third of respondents still name complexity as a top challenge ten years into the ecosystem. Tools that solve problems which did not exist before the tools arrived: that pattern has a paper trail long enough to count as a tendency, and it is quoted here as exactly that.

The bench, in four figures 0 ms capability mode, measurable cost 0.17 s jail, cold start 151 packages inside nginx:latest 144 MB daemon pair, idle (quoted)

On its own terms the adjective holds: a container is lighter than a virtual machine, and alpine is lighter than debian. The fence itself, on either kernel, costs nothing a bench can find; that part of the legend was true, and it was never the part being sold. The bill begins where the product begins: 151 packages in the parcel, a metered counter in front of every install, a root daemon keeping 144 megabytes warm, a licence check at the office door, and an update cadence set by somebody else's base image.

Docker monetised a political problem and left the technical one where it stood. On the kernel that never had the political problem, this morning's entire apparatus came to two packages, ten megabytes and a fifth of a second.

The future of Linux containers, as announced in Santa Clara, had at that moment been shipping for thirteen years. The invoice was for everything around it.

Measurement data and commands

capbench.c   200,000 x 4 KiB write+read against a /tmp file;
             unrestricted vs cap_rights_limit(CAP_READ|CAP_WRITE|CAP_SEEK)
             + cap_enter(); cc -O2; five runs per variant, medians reported
jail runs    jail test_and_benchmarks, FreeBSD 15.0-RELEASE, rolled back
             to a golden ZFS snapshot before each series; base system only
jail start   service jail start, three runs: 0.18 / 0.18 / 0.17 s
registry     auth.docker.io token + registry-1.docker.io manifest API for
             nginx:latest and nginx:alpine (amd64), compressed layer sizes
             summed; no Docker involved
census       all seven nginx:latest layers fetched via the blob API and
             unpacked in the jail; packages counted from /var/lib/dpkg/status,
             executables counted under bin and sbin paths; unpacked disk size
             discarded as not proven, the dpkg ledger counts stand
freebsd      pkg install nginx in the fresh jail, timed; service nginx
             onestart; resident size via ps -o rss
bench        Hetzner AX41-NVMe, AMD Ryzen 5 3600, 64 GB, NVMe,
             FreeBSD 15.0-RELEASE, measured 25 August 2026
not here     Docker's own runtime on this hardware; the idle-daemon figure
             and the whole running tab are quoted, sources below