The Bench Marker ■ measured, not assumed
The claim under examination is one of those sentences nobody bothers to defend any more, because everybody has heard it often enough to mistake it for a finding. Clean, fast software distribution needs an ecosystem: an image format, a registry to hold the images, a daemon to unpack them, and something on top to decide where they go. Anything less is said to be artisanal, which in this trade is not a compliment.
So I weighed it against the arrangement a filesystem gives you for nothing, on a system where ZFS is underneath and the isolation is already in the kernel. Weighing an ecosystem against thin air would prove very little and flatter it enormously.
Anybody can repeat two of the three measurements below from a laptop in about five minutes, using nothing but published catalogues. The third came off a real migration under real load, which is a nicer bench than anything I could have staged.
What the parcel weighs
Start with a web server, because everybody has installed one and nobody thinks about the packaging.
On FreeBSD, pkg fetches nginx at 602,576 bytes and hands you 1,715,152 bytes installed. It brings exactly one dependency, pcre2. The whole tree, server and dependency together, is 2,128,456 bytes over the wire.
From a registry, nginx:latest for amd64 arrives as seven layers totalling 66,322,314 compressed bytes, on a base image of debian:trixie-slim. Both figures are compressed, so neither side is being flattered. That is a factor of thirty-one for the same web server.
Now the tool that is supposed to make fleets manageable. Ansible, from the same catalogue, pulls 23 packages, 96,279,253 bytes over the wire, and settles at 834,543,778 bytes installed. Its core alone accounts for 315,173,383 of that.
Hold that against the base system it configures. The whole of FreeBSD 15.0, kernel and userland in one archive, is 165,127,228 bytes. The configuration management tool arrives at fifty-eight per cent of the weight of the operating system it manages, which is the sort of ratio that usually indicates somebody has lost the thread.
And here is the figure that made me go back and check the catalogue twice. The Ansible module that writes a line into rc.conf on FreeBSD, ansible-sysrc, weighs 9,218 bytes installed. Nine kilobytes of glue, wrapped in eight hundred megabytes of apparatus, and at the end of it the module calls sysrc, which has been sitting in the base system since 2013 and does the job on its own.
Whether that is absurd depends entirely on what else the eight hundred megabytes are doing for you, and we will come to that, because the answer is not nothing.
The fleet that moved house
Sizes are the easy part. The question with real money attached is what happens when the machines actually have to move, so consider a migration that happened last week rather than a scenario I invented.
Sixteen jails, each one a service, living on a rented box with sixty-four gigabytes and a mirrored pair of NVMe drives. They were moved to a second rented box roughly a thousand kilometres away, over the public internet, as a single replication stream. Eighteen datasets in total once you count the base and the parent, holding 18.7 GiB of actual data.
The whole fleet arrived in seven minutes flat, at about 45 MB/s onto disk at the far end.
zfs snapshot -r rpool/jails@move-1
ssh -6 operator@old-host 'sudo zfs send -R rpool/jails@move-1 | zstd -T0 -3 -q' \
| zstd -d -q | sudo zfs receive -s -u rpool/jailsFourteen of those eighteen datasets are clones of one base dataset, and this is the bit that answers the objection everybody raises first. A store of prepared images, they say, will drown you in copies: one for each combination of software, all of them nearly identical, all of them costing you disk. It does not, because the clone relationships survive the stream. Fourteen services share one base system, they shared it before the move, and they were still sharing it when they arrived. The common part crossed the wire once.
The local figures explain why the network was the slow part. Reading a dataset out and piping it to wc -c runs at 1.8 to 2.2 GB/s depending on which compression the pool uses. A full local send | receive of 6.18 GiB logical took 5.14 seconds, so 1,232 MB/s, and receive is the expensive half because the far end recompresses and writes into a mirror. Compressing the stream with zstd -3 for the wire gives 2.88 to 1 and runs at about 1.2 GB/s in. Nothing in that chain is the bottleneck except the line itself, which is rather the point.
Then the switchover, on the following day. Everything had already crossed. What remained was the delta between two snapshots: 29 megabytes, and with services stopped and started again, two minutes and twenty-two seconds. That is the entire cut-over for sixteen production services.
There is one more property in that stream, and I had it filed under nuisance until the person who ran the migration corrected me. A recursive send carries all 192 rotation snapshots along with the data, which is why 18.7 GiB of payload produced a 48.8 GB stream. I had written that down as a trap for the unwary. It was deliberate. The entire backup history moved to the new machine along with the services, and the clone relationships came too. A push to a registry moves a state and leaves the past behind on the old shelf.
The interface that stopped moving
Here is where the honest objection turns up, and it is a good one.
Clones assume the machines are yours to build. They assume one operating system, and if not one version then at least versions that agree with each other about how things are configured. The moment your estate is mixed, a clone is no help at all, and this is precisely the ground on which a configuration management tool earns its money: it makes different systems, at different patch levels, on different platforms, end up in the same state. Neither a clone nor a shell script can do that. It is a real capability and it is worth paying for.
So the fair question is how big that gap actually is on this platform. That one turned out to be measurable too, and as far as I can tell nobody had bothered.
The default rc.conf shipped with each release lists every variable the system understands. Pull it from the source tree at each release tag and see how many of those names still work today.
Eighty-four per cent of the configuration surface from 2012 is still valid in 2026. Over the last three years, ninety-eight.
The sixty-six casualties are worth reading as a list, because they are not breakages. atm_enable and atm_pvcs, for ATM networks. ipxrouted_enable, for Novell IPX. ibcs2_enable and svr4_enable, for binary compatibility with commercial Unixes that no longer exist to be compatible with. Nine named_* variables, from when BIND lived in the base system. gbde_*, superseded by GELI. Somewhere in there, timed. The interface held its shape. What went away were the things it used to point at.
Which puts a number on the gap the tool exists to bridge. On this platform, across fourteen years, that gap is sixteen per cent, and across three years it is under two.
Where it does not reach
Two limits, and neither is small.
The first is that both ends must speak ZFS. A replication stream is not a portable artefact; it goes to your own fleet or it goes nowhere. An image in a registry can be handed to a stranger, and a stranger can run it, on a platform you have never seen. That is a genuine advantage and no amount of measured throughput makes it go away.
The second is that the fast numbers stop at the edge of your own network. The same 18.7 GiB, compressed to 11.9 and pushed to remote backup storage, moved at 3.6 MB/s and took the better part of an hour. The far end was the constraint rather than the filesystem, since the same route carries 13 to 20 MB/s per channel when something else is receiving. Still, if the story is going to be told honestly, it includes the hour.
And the prepared datasets do not maintain themselves. Somebody builds them, and a vulnerability in a library that sits in eight images is a vulnerability in eight images. The work moves to the front and becomes visible, rather than disappearing into every run.
The name
Which leaves the question of why the older and lighter arrangement never took the market, and part of the answer has nothing to do with engineering at all.
Poul-Henning Kamp's paper from 2000 is called "Jails: Confining the omnipotent root". Read as security, the name is exact, because the job was to lock root in a room. Nobody was thinking about distribution, and the word carries none. Thirteen years later the same kernel property was announced elsewhere under a shipping metaphor: a container, a standard box, something you load and send, the machinery of world trade in a single noun. Both names describe the same confinement truthfully, and one of them sounds like the parole board.
The measurements above are what the badly named version does. Two megabytes for a web server rather than sixty-six. Sixteen services across Europe in seven minutes, sharing one base between fourteen of them because the clone survived the journey. A configuration surface that has drifted sixteen per cent in fourteen years. No daemon, no registry, no pull limits, and no small kerfuffle at three in the morning when the thing that fetches your images has decided today is not the day.
The same nginx tag that measured 63,327,223 bytes a fortnight ago measures 66,322,314 today. Same name, three megabytes heavier. Nobody was told. A stream sends the blocks that are there, and asking for it twice gets you the same machine both times, which is a modest sort of promise until you have spent an afternoon working out why staging and production are not the same.