Vivian Voss

bhyve Runs a VM in One Process

technical beauty scope freebsd bhyve virtualisation

On 14 October 2026 FreeBSD published an advisory for its hypervisor, bhyve. Several flaws had been found in the code that pretends to be hardware for a guest. Most of them sat in the emulated USB 3 controller and the emulated sound card. The section headed Workaround is a single sentence long, and it says more about the design than most architecture papers manage: "Virtual machines that provide none of the "xhci", "hda", and "fbuf" devices to guests are not affected."

A server guest with a disk, a network card and a serial console has none of those three. Its administrator read the advisory, checked the command line of the guest and went back to work. That is what a short list of devices is for.

bhyve came from Neel Natu and Peter Grehan, who wrote it at NetApp, and its manual records that it "first appeared in FreeBSD 10.0", released in January 2014. It virtualises the host's own processor and has never tried to imitate anyone else's.

One process, as it should be

The split is the usual one for a modern hypervisor. A kernel module, vmm, drives the processor's virtualisation extensions and handles what must happen in the kernel. Everything a guest sees as hardware runs in user space, in one ordinary bhyve process per guest, which ps lists and top measures like any other program.

QEMU with KVM is built the same way, one process per guest on top of a kernel module, and on that point there is nothing to choose between them. The difference lies in what that one process carries, and in what the operating system around it adds.

Nineteen devices

bhyve(8) lists the device models a guest can be given, and the list on FreeBSD 15.1 runs to nineteen: the host bridge and PCI passthrough, seven VirtIO devices for network, block storage, SCSI, shared folders, randomness, console and input, three flavours of AHCI for disks and optical drives, NVMe, an e1000 network card for guests without VirtIO drivers, a serial port, the LPC bridge, a framebuffer, a USB 3 controller and an HDA sound card. That covers a server comfortably and gives a desktop guest its bits and bobs as well. There the list stops.

Every one of those models is code that reads values a guest controls, and the guest may be hostile. That is the reason the list is short, and the advisory history bears it out: of the sixteen FreeBSD advisories with bhyve in their name since 2016, the recent ones sit in exactly those models, NVMe, VirtIO, HDA and xHCI, and the October workaround spells out that a guest without the optional ones never met the flaw at all.

bhyve and QEMU, measured the same way Source lines without comments bhyve userland 46,709 QEMU, all targets 1,969,575 Installed size FreeBSD-bhyve 664 KiB qemu-system-x86 64,358 KiB freebsd-src releng/15.1, qemu-11.1.3, FreeBSD 15 base packages, Debian 13 index; 6 November 2026

QEMU takes the other view and emulates a great deal. Its source tree for release 11.1.3 still carries a floppy disk controller in hw/block/fdc.c, and in 2015 that controller became VENOM, CVE-2015-3456, which the NVD describes as a flaw by which "local guest users" could "possibly execute arbitrary code", on hosts that had never seen a floppy. Each model a hypervisor does without is one place fewer for that kind of flaw to live, and bhyve's own advisories show that the models it keeps are precisely where its flaws have turned up.

The process locks its own door

Before the guest runs a single instruction, bhyve opens every device file it will need and then calls caph_enter() (line 1000 of bhyverun.c in releng/15.1). From that moment the process is in Capsicum capability mode: it keeps the descriptors it already holds, with the rights it trimmed them to, and it can open nothing new on the host.

The security team rely on it in writing. The advisory of September 2024 for a heap overflow in the USB code describes the worst case and then adds: "Note that bhyve runs in a Capsicum sandbox, so malicious code is constrained by the capabilities available to the bhyve process." QEMU offers a seccomp sandbox through its -sandbox option, and libvirt adds SELinux or AppArmor profiles on the distributions that ship them; on FreeBSD the confinement is in the program itself, and it is on by default.

A guest inside a jail

The second wall is the jail, and here bhyve fits FreeBSD rather well. jail(8) has a parameter for it, allow.vmm, described as "The jail may access vmm(4)", and a separate one, allow.vmm_ppt, for jails that may hand physical PCI devices to their guests. The kernel checks both in vmm_priv_check(). A guest can therefore run in a jail of its own with its own network stack, inside which the bhyve process then locks itself into Capsicum as well.

How a bhyve guest starts jail with allow.vmm open devices zvol, tap, console caph_enter() Capsicum from here on guest runs through held descriptors after caph_enter the process can open nothing new on the host one bhyve process per guest nineteen device models at most ZFS volume snapshot and clone for the next guest

The disk comes from ZFS. A guest disk is usually a ZFS volume, /dev/zvol/tank/vm/web, so a snapshot before an upgrade is one command and a new guest from a prepared image is a zfs clone, which costs no space until the two start to differ. Starting a headless server guest is a doddle:

bhyve -c 2 -m 4G -H \
  -s 0,hostbridge \
  -s 3,nvme,/dev/zvol/tank/vm/web \
  -s 4,virtio-net,tap0 \
  -s 31,lpc -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  web

Five devices, one of them the console, none of them on the October list. The firmware line points at the UEFI image from the ports collection; a FreeBSD guest can skip it and start through bhyveload.

The design has travelled, too. xhyve, which describes itself as "a port of bhyve to macOS", carried the same device models onto Apple's Hypervisor.framework, which says something about how cleanly the user-space part separates from the kernel underneath.

The sheet

Efficiency. Source in releng/15.1 (commit 95719d8), lines without comments and blank lines: usr.sbin/bhyve 46,709 in 161 files, of which the amd64-specific part is 7,198, arm64 935 and RISC-V 827; the kernel module in sys/amd64/vmm 20,720 and sys/dev/vmm 2,255; libvmmapi 2,166; with bhyvectl and bhyveload, 74,674 in all. The package FreeBSD-bhyve 15.1p5 installs 679,988 bytes and depends on nothing outside the base system. QEMU 11.1.3, counted the same way without tests, firmware, bundled subprojects, documentation and scripts, comes to 1,969,575 lines in 5,583 files, for all of its targets; the x86 target alone is 57,039 and the device models in hw/ 644,480. On Debian 13 qemu-system-x86 installs 64,358 KiB, plus 18,494 for qemu-system-data.

Security. Of the four questions: bhyve usually runs as root; it takes no data from the network itself but reads everything a guest writes to its devices; it parses device protocols of nineteen models; and it gives up the right to open anything new through Capsicum before the guest starts. Sixteen advisories since 2016 carry bhyve in their name, the recent ones in device models, and the October 2026 one names the configuration that was safe.

Longevity. In the base system since FreeBSD 10.0, January 2014, maintained there ever since, now with code for three architectures, and carried to macOS as xhyve on bhyve's own device models.

Stability. A guest is still described by -s slot,device,options as in the first release; a configuration file read with -k has since joined the command line, and a headless server needs five slot lines.

Measured on 6 November 2026: lines counted with a script that strips C comments and blank lines, over freebsd-src releng/15.1 and qemu-11.1.3.tar.xz; package sizes from packagesite.yaml in FreeBSD:15:amd64/base_release_1 and Debian 13's Packages index for amd64; device models from bhyve(8); advisories counted on the FreeBSD security page.

The limit

bhyve virtualises the host's own processor. That is the job it was built for, and the source tree now carries it for three architectures. A guest written for another processor is an emulator's task, and the ports collection carries QEMU for exactly that, one pkg install away on the same machine.

Linux and Windows guests boot through one package. pkg install edk2-bhyve provides the UEFI image the example above points at; FreeBSD guests start through bhyveload and need nothing extra.

A large fleet of guests gets its manager from the ports. The base system starts each guest with one command. vm-bhyve from the ports adds templates and autostart for anyone running dozens of them. The disks sit on ZFS either way.

Capsicum and the jail are there for a process that starts as root. bhyve usually does, which is why it confines itself before the guest runs and why the device list stays short.

A hypervisor is mostly the hardware it pretends to be, and bhyve pretends to be very little of it.