There is a line on the Amazon bill for every Kubernetes cluster, and for the first fourteen months it reads $0.10 an hour. Then the version the cluster runs leaves what Amazon calls standard support, and the same line reads $0.60. Over a year (8,760 hours, leap years aside) that is $876 against $5,256, for the same control plane doing the same work.
Nobody has to ask for the higher rate. Amazon's documentation says so in one sentence: "Extended support is enabled by default for new clusters, and existing clusters." The next paragraph adds the part that turns it into a meter: "Once your cluster has entered extended support, you cannot disable it." Google sells the same thing for its own clusters, an extra $0.50 an hour on top of the usual $0.10, so the market has settled on a price for not upgrading.
The fee has its origin upstream, and Amazon is entitled to it. Kubernetes publishes a new version roughly three times a year and gives each one a year of patches. Amazon keeps older versions alive for 26 months instead and backports the security fixes itself, and that work costs money. The more interesting question is why so many teams would rather pay a fair whack to wait than take the upgrade for free.
What an upgrade swaps
Kubernetes keeps a migration guide for the parts of its API it has switched off. I counted them this morning: between version 1.16 and version 1.32 the guide lists thirty API versions that are "no longer served", twelve of them in 1.22 alone. In 1.25 PodSecurityPolicy went, and with it "the PodSecurityPolicy admission controller will be removed", which was how a good many clusters had kept their pods in line. Every one of those removals is a manifest somebody has to rewrite before the upgrade goes through.
Beneath Kubernetes sits the machine that runs it, and that changes too. On 26 November 2025 Amazon stopped publishing node images based on Amazon Linux 2, and the successor, Amazon Linux 2023, brings "the next generation of unified control group hierarchy (cgroupv2)" and requires the newer metadata service by default. Software that read the old cgroup layout has to be told about the new one. So an upgrade on a managed cluster moves several layers at once, each at its own pace (the Kubernetes API above, the node's distribution and its kernel interface below), and any of them can be the one that falls over on a Tuesday afternoon.
That is the fee, then. It is the price of the time a team needs to find out what changed under the bonnet this round. To see why there are so many layers to change, it helps to go back to the year the job was first done in one.
The job, done in 2000
FreeBSD 4.0 shipped the jail in March 2000. A jail is one object in the kernel: a process tree with its own root directory and its own addresses, whose root user is root only inside it. One system call creates it, and the kernel enforces every boundary at once, because they are all the same boundary.
Linux professionals saw this and wanted the same, which is a compliment, and about the nicest one a kernel can be paid. Over the next eighteen years the Linux kernel gained the same isolation in separate parts, one at a time. The manual pages for clone(2) and unshare(2) give the dates: mount namespaces in Linux 2.4.19 (2002), IPC and hostname namespaces in 2.6.19, network and process namespaces in 2.6.24, user namespaces usable from 3.8, cgroup namespaces in 4.6, and time namespaces in 5.6, in 2020. Resource control came separately as cgroups in 2.6.24, and its own manual page is candid about how that went: "the development of these controllers was largely uncoordinated, with the result that many inconsistencies arose". A second version had to be written, made official in Linux 4.5, and that second version is precisely what Amazon Linux 2023 now asks your software to understand.
Why Linux had to rebuild the idea at all is an older story. FreeBSD's ancestors spent the early nineties in court, in a lawsuit between AT&T's Unix company and Berkeley that was settled only on 4 February 1994, and in those years the attention went to a kernel a student in Helsinki had written. Linus Torvalds said it himself in 1993: "If 386BSD had been available when I started on Linux, Linux would probably never have happened." More than one Linux kernel developer has told me the same thing over the years, in nearly those words: they chose Linux because it was not in court, and they rather preferred BSD.
The family
The result of all that careful work was a set of building blocks that nothing in the kernel ties together. Something outside has to assemble them into a container, and in March 2013 that something became Docker.
Docker answered a genuine need, the one a fragmented Linux had created. A Linux service had to run on many distributions, each with its own libraries and habits, and an image that carried its own little distribution ran the same everywhere. Docker's own announcement of version 0.9, in March 2014, describes the job in detail. Its new library, libcontainer, could "manipulate namespaces, control groups, capabilities, apparmor profiles, network interfaces and firewalling rules", and its great virtue was that it "insulates Docker from the side-effects introduced across versions and distributions of LXC". The same post lists the isolation tools Docker hoped to drive one day, and BSD Jails are on the list.
Node.js then made a lot of containers seem necessary. Its documentation is clear about the design: "Node.js has two types of threads: one Event Loop and k Workers", and JavaScript runs on the one. Node has offered the cluster module for years and worker_threads since version 10.5, so a single Node process can use more than one core. In practice, though, the habit became one process per core and one container per process, and a modest web service turned into a good many containers that someone had to keep track of.
Keeping track was the next child. Kubernetes arrived in June 2014 to schedule all those containers across machines, and it brought a release rhythm of its own along with an exam: the Certified Kubernetes Administrator certificate costs $445. Each image in the cluster still carries a distribution of its own, chosen by whoever built it. Sysdig's report for 2023 found that "87% of container images have high or critical vulnerabilities", which is par for the course when every service brings its own copy of an operating system.
Who collects and who pays
The platforms collect twice. Every EKS cluster pays $0.10 an hour to exist, and every cluster that cannot keep up with three releases a year pays six times that. Around them stands an industry of courses and consultants, which earns its money from the complexity and has no particular reason to wish it away.
The operator pays, and in more than the fee. Cast AI measured more than 2,100 organisations' Kubernetes clusters across 2024 and found an average CPU utilisation of 10 per cent and memory utilisation of 23 per cent. Most of the machines a team rents for its cluster are, on average, waiting. On top of that comes the upgrade work itself, layer by layer, every few months, and the fee for every month it slips.
The same job on FreeBSD
On FreeBSD the job has one layer, and the jail is still the same kernel object it was in March 2000. A service gets a jail through a few lines in /etc/jail.conf and starts with service jail start, with no daemon in between and no control plane to rent. Several hosts are run with what the base system brings anyway: zfs send moves a jail and its data from one machine to the next, and CARP lets a neighbour take over an address when a host goes down.
What makes the difference on upgrade day is a promise the project writes down. The Committer's Guide puts it in two sentences: "The FreeBSD-STABLE branch must maintain ABI and API compatibility with earlier versions of that branch. Do not merge changes that break this compatibility." The Handbook adds a principle with a name, POLA, the principle of least astonishment: "As FreeBSD evolves, changes visible to the user should be kept as unsurprising as possible." Its example is wonderfully small. Merely rearranging the startup variables in /etc/defaults/rc.conf would count as a violation.
The branch that carries that promise is long-lived. FreeBSD 15 is supported until 31 December 2029, four years from its first release, and nobody charges for any of them. Within that time a minor upgrade keeps the ABI, so programs built for 15.0 go on running, and freebsd-update can create a boot environment first, so the old system is one reboot away if something does go wrong.
The limit
Kubernetes manifests do not run on FreeBSD. A team that lives in YAML moves out service by service: each one gets a jail, its configuration goes into jail.conf and rc.conf, and its data into a ZFS dataset that zfs send can carry to the next host. Whoever wants a scheduler across many machines finds Nomad with a jail driver in the ports, though a fleet with a tenth of its CPU busy rarely needs one.
A major version still needs its upgrade. FreeBSD plans a new major release every two years and supports each branch for four, so a team upgrades on its own schedule, within the years in which two supported branches overlap, with the boot environment as the safety net.
The support period has become shorter. In 2024 FreeBSD cut the life of a stable branch from five years to four, so that the security and ports teams look after no more than two branches at once. Four years without a fee still compares rather well with fourteen months before the meter changes.
On FreeBSD the only meter in the server room is the one on the wall, and it measures electricity.