Vivian Voss

buildworld Builds All of FreeBSD

unix universe freebsd buildworld hardening
# cd /usr/src
# make -j8 buildworld buildkernel

Two words after make, and what follows is the construction of an entire operating system from one directory. The tree under /usr/src carries its own compiler, and the build makes that first; then the C library, the shell, ls, pf, zfs, OpenSSH, the boot loader, and finally the kernel, all from the same checkout, with the same switches, in one run. When it finishes, make installkernel puts the kernel in place. make installworld follows with the rest, after which etcupdate merges the configuration files. Then the machine reboots into a system that was compiled on it, for it.

I took the system that most people never compile, the stock one, and counted what is in it. FreeBSD 15.0-RELEASE for amd64 ships its base system as base.txz, 165 MB packed, 14,631 files unpacked. 777 of those files are programs. 772 of them are position-independent executables, so ASLR can move them around, and all 777 carry RELRO, so part of their relocation table turns read-only after loading. The number that caught my eye was a different one: of 777 programs, none is linked with BIND_NOW. Without it, the table of addresses a program uses to reach its libraries stays writable while the program runs, which is a gift to anyone who has found a way to write somewhere they should not. And 27 files in the archive carry the setuid or setgid bit, which means they start with privileges their caller lacks.

base.txz, FreeBSD 15.0-RELEASE amd64, as shipped 777 programs 772 position-independent 0 linked with BIND_NOW 27 setuid or setgid files all 777 programs carry RELRO; counted with pyelftools on 26 October 2026

This is what a default looks like when it has to suit every machine at once.

One tree, one make

build(7), the page that documents the targets, has been in FreeBSD since 4.3, and the tree it describes holds the whole system. buildworld builds "everything but the kernel", buildkernel builds the kernel and its modules, and the target called toolchain creates "the build toolchain needed to build the rest of the system", which is the polite way of saying that the compiler is part of the tree it compiles.

Two files steer it. /etc/make.conf holds what applies to every compilation, and its CPUTYPE variable "controls which processor should be targeted for generated code", so a machine with a newer processor can have its whole base system built for exactly that processor. /etc/src.conf holds what applies to the base system, and in 15.0 it knows 259 switches, each of the form WITH_SOMETHING or WITHOUT_SOMETHING. Some switch mitigations on. Others decide whether an entire component is built at all.

What a binary distribution decides for you

A binary distribution compiles once, for everyone, and has every reason to. Its packages must run on the oldest processor it supports and on the newest, with every optional feature a user might want, because the alternative is a separate build for every taste and a support matrix nobody could staff. For millions of machines that is a sensible trade. The cost is simply that every decision has been taken already: which processor the code is tuned for, which mitigations are on, which components are present whether anyone uses them or not, and which privileged programs come with them.

Gentoo, on the Linux side, builds from source as a way of life. Its USE flags switch features per package across the whole installation, and its hardened profile builds every package with stack clash protection, immediate binding and _FORTIFY_SOURCE=3 by default. A Gentoo administrator reading this far will say that none of it is news.

The difference sits one level up. emerge @world assembles a system out of a great many independent projects, each with its own build system (autotools here, CMake there, Meson, Cargo), each with its own release schedule, stitched together by the distribution's recipes. make buildworld builds one project. The kernel, the C library, the firewall and the compiler live in the same tree, are tested together and carry the same version number, so a switch in src.conf reaches all of them in the same run, and what comes out is a system the project would recognise as its own, merely trimmed and tightened.

The receipt, in lines of src.conf

The count above follows from the defaults. In 15.0 PIE, RELRO, SSP and reproducible builds are on by default, according to bsd.opts.mk in the tree; BIND_NOW is off, which matches the count, and so are the retpoline and register-zeroing mitigations. Three lines change that for the entire base system:

WITH_BIND_NOW=yes
WITH_RETPOLINE=yes
WITH_ZEROREGS=yes

The first gives "'full' Relocation Read-Only (RELRO) support", in the words of src.conf(5): "the entire GOT is made read-only after performing relocation at startup, avoiding GOT overwrite attacks." The second builds the system with the retpoline mitigation against Spectre variant 2, and the third zeroes the registers a function used before it returns, which, the manual adds, "reduces the number of usable ROP gadgets for attackers." Each applies to every program in the tree at once, the shell and the boot loader included.

The setuid files go a step further, because the safest privileged program is the one that was never compiled. A server that prints nothing and dials nowhere has no use for lpr, ppp, quota or btsockstat, and seven lines in src.conf remove ten of the 27:

WITHOUT_LPR=yes
WITHOUT_AT=yes
WITHOUT_PPP=yes
WITHOUT_QUOTAS=yes
WITHOUT_BLUETOOTH=yes
WITHOUT_SENDMAIL=yes
WITHOUT_AUTHPF=yes

I traced each of them to its line in the tree's Makefiles (SUBDIR.${MK_LPR}+= lpr and its neighbours), and with lpq, lpr, lprm and lpc gone along with at, ppp, quota, btsockstat, sendmail and authpf, those ten never reach the disk. There is nothing to patch in a program that was never built, and nothing for an attacker to call.

Seven lines in src.conf, ten privileged programs never built WITHOUT_LPR lpq lpr lprm lpc WITHOUT_AT at WITHOUT_PPP ppp WITHOUT_QUOTAS quota WITHOUT_BLUETOOTH btsockstat WITHOUT_SENDMAIL sendmail WITHOUT_AUTHPF authpf 27 setuid or setgid files in the stock base; 17 remain

The result, mind, never leaves the package system. make packages, run after the build, creates "a freebsd-base(7) package repository containing packages that can be used to install or upgrade the base system", and the Handbook spells out the sequence as make buildworld && make buildkernel && make packages, followed by an ordinary pkg repository on a web server for the other machines. From there pkg handles the hardened, processor-tuned base system as it would any package.

From one tree to pkg src.conf · make.conf switches, CPUTYPE buildworld · buildkernel toolchain first, then all installkernel · installworld this machine make packages freebsd-base repository Poudriere ports, same make.conf, signed pkg install · pkg upgrade every other machine, base system and ports alike

The software on top follows the same route. Poudriere builds ports in clean jails from the ports tree with whatever make.conf the administrator gives it, and the ports framework reads WITH_PIE, WITH_RELRO, WITH_BIND_NOW, WITH_FORTIFY, WITH_STACK_AUTOINIT and WITH_ZEROREGS there, so the applications get the same CPUTYPE and the same hardening as the base. It signs the resulting repository with the key named in PKG_REPO_SIGNING_KEY, and pkg-repo(8) accepts rsa, ecdsa, eddsa or mldsa signatures, the last of them a post-quantum scheme. And the project's own packages keep working beside all of this, because they are built "on the oldest supported minor release within each major branch to ensure ABI and KBI backwards compatibility".

A home-built world on 15.0 is still 15.0 as far as pkg is concerned.

The limit

Building costs a fair bit of time. A world and a kernel are an operating system and a compiler, and they take as long as that takes on the machine at hand. I measured the archive and leave the build time to the reader's own machine.

Updates become your job. In the Handbook's own words, "Only the GENERIC kernel can be automatically updated by freebsd-update". Whoever builds a custom system rebuilds it for each security fix, or serves it from their own base package repository as described above.

Gentoo goes further at the compiler. Its hardened profile sets _FORTIFY_SOURCE=3 and stack clash protection for everything by default. On FreeBSD the ports framework offers FORTIFY as a switch at level 2, src.conf in 15.0 has no switch for it in the base system, and stack clash protection appears in neither list. Gentoo's USE flags are also finer than src.conf, because they work per package.

A removed component is removed for everyone. A port that wants sendmail or at will not find it, so the seven lines above suit a server whose duties are known, and the list for a workstation looks different.

The bill

Today's default on a binary distribution: the compiler flags, the processor target, the mitigations and the set of privileged programs were chosen once, by somebody else, for a machine that is not yours, and every machine receives the same answer. The Linux alternative builds a great many projects one by one, each by its own rules.

On FreeBSD the bill is a file of ten lines and the time it takes to compile what they describe. In return: an entire operating system, kernel to compiler, built for this processor, with immediate binding in all 777 programs, ten privileged binaries that never exist, and a repository that pkg installs and upgrades as though nothing unusual had happened.