Sun Microsystems in 1984 had a problem that was mostly furniture. The company sold workstations, the workstations sat in rooms full of engineers, and engineers do not stay at one desk. Somebody would finish a shift on the machine by the window, come in the next morning, and sit down at whichever one was free. Their files were on the machine by the window.
The obvious answers were all bad. Copy everything overnight and hope nobody edits the same file twice. Give each person one machine and let it idle when they are elsewhere. Or teach every single program on the system how to fetch a file over a network, which means changing cat, vi, the compiler, the linker, and everything anybody might ever write afterwards.
What Sun did instead was slide the network one layer below the file system, where no application would ever have to look at it. A directory on another machine gets attached to the local tree, and from that moment the kernel answers requests for it by talking to the other machine. Nothing above the kernel knows. cat opens a file and reads bytes from it, exactly as it would on a local disk, and the fact that the bytes travelled over an Ethernet is somebody else's business.
That is the whole idea, and it is the reason the thing is still here.
There was a second decision, and it mattered as much as the first. In 1989 Sun published the protocol as RFC 1094, which meant anybody could implement it without asking permission or paying for a licence. A commercial workstation vendor giving away the wire format of its own product was not the obvious move, and the reasoning was straightforwardly self-interested: a protocol that only Sun machines speak is worth less than one that every machine speaks, because customers buy the machine that talks to everything else in the building. The effect outlived the reasoning. Sun was bought in 2010 and the brand has since been retired, and the thing they gave away is now in every operating system worth the name, along with the invoices in the last section of this piece.
What it actually solved
The trick lay in the decision about where to put the network code.
Every alternative on offer required applications to know about the network. Once they know, they have to handle it: connection failures, retries, partial reads, authentication, all of it in every program separately, done slightly differently each time. NFS pushed the whole business one layer down and gave applications the interface they already spoke. Open, read, write, close, and a path that starts with a slash.
The protocol was deliberately stateless in its early versions, and that choice came from watching real machines. Sun assumed servers would crash, because in 1984 they did. A stateless server holds no memory of who has what open, so when it comes back up there is nothing to reconstruct; the client repeats its last request and carries on. A file server rebooting stopped being an incident.
There is an irony in that which is worth having. Statelessness cannot lock a file, because a lock is state by definition, and the early versions had to bolt locking on the side through a separate daemon that everybody found awkward. NFSv4 took the state back into the protocol, with sessions and delegations, and thereby gave up the property that made version two elegant. The industry has been having the same argument about stateless services ever since, mostly without noticing that a file protocol worked through it forty years earlier.
Forty years, and the meter starts
The chronology is worth laying out, because the interesting part sits at the end of it, where the line between a protocol and a product stops being obvious.
What it still does well
On the machine in front of me, the whole thing weighs 1.73 megabytes.
# ls -l /boot/kernel/nfs*.ko
-r-xr-xr-x 1 root wheel 229944 nfscommon.ko
-r-xr-xr-x 1 root wheel 596616 nfscl.ko
-r-xr-xr-x 1 root wheel 544360 nfsd.ko
-r-xr-xr-x 1 root wheel 138776 nfslockd.ko
# pkg which /sbin/mount_nfs
/sbin/mount_nfs was not found in the database
# pkg which /usr/sbin/nfsd
/usr/sbin/nfsd was not found in the databaseThat covers the client, the server, the lock manager and the automounter, with six userland programs adding another 219,608 bytes. For comparison, in the same kernel build, smbfs.ko is 235,400 and fusefs.ko 207,488.
None of it belongs to a package. Both files arrived with the system, along with seven manual pages covering exports(5), mount_nfs(8), nfsd(8), mountd(8), autofs(4), auto_master(5) and nfsv4(4). Exporting a dataset is a property rather than a configuration file:
# zfs set sharenfs=on tank/workThe pool tells the daemon what to publish, and the two facts live in one place instead of two. It is a small thing that shows the difference between a system built as one tree and a system assembled from parts: the filesystem knows it is shared, the share knows which filesystem it belongs to, and there is no third file that can drift out of step with either. On a machine where storage and sharing come from different projects with different release cycles, that arrangement has to be maintained by hand, and it is maintained by hand roughly as well as anything else that has to be maintained by hand.
The manual pages deserve a word too, because they are the part people skip. Seven pages, written by the same team that wrote the code, versioned in the same tree, describing the behaviour of the thing actually installed rather than the behaviour of whatever version the documentation site happens to be showing today. When exports(5) says a flag does something, the flag does that thing on this machine. It is a mundane property and it is worth more than most features.
What all of that adds up to is POSIX semantics across machines. What you get for that is POSIX semantics across machines. Two hosts can hold the same file open. A process can write to the middle of a file without reading the rest of it. rename is atomic. Directory listings are directory listings. Locks exist and other machines respect them.
That list sounds unremarkable until you build the same thing on top of an object store.
None of this makes an object store a bad thing. It makes it a different thing, aimed at a workload where objects are written once and read by many, and where nobody needs to open the same file from two machines at once. The trouble starts when a team reaches for it because it is the default in their platform, and then spends a fortnight rebuilding a shared directory out of it.
Where it asks for trouble
Being fair to a protocol means saying where it hurts, and this one has a few sore spots that anybody who has run it will recognise.
Start with authentication, where the honest description is that AUTH_SYS does not authenticate. The client tells the server which user identifier and group identifiers it is acting for, and the server takes it on trust. If the numeric identifiers on the two machines do not line up, somebody ends up with somebody else's permissions, and the fix is the sort of faff nobody enjoys: keeping identifiers synchronised across every host that mounts the share. The proper answer has existed for years, which is Kerberos through RPCSEC_GSS, and it is deployed far less often than it should be because setting up a realm is a project rather than an afternoon.
Then there is failure behaviour. A hard mount does what it says: when the server stops answering, processes touching that mount wait, and they wait in a state where they cannot be interrupted easily. One unreachable file server can freeze a room full of machines that were only listing a directory. Soft mounts return errors instead, which sounds better until an application that was never written to expect a failed read writes half a file and calls it done.
The failure mode has a signature that anybody who has met it recognises immediately. The machines are not slow; they are stopped. Load average climbs while the processors sit idle, because the processes are not computing anything, they are waiting in the kernel for a reply that is not coming. ls in the wrong directory never returns. The usual escalation is somebody rebooting a workstation that was working perfectly, because the thing that broke is three racks away and belongs to another team.
Latency comes last, and it is arithmetic. Every operation costs a round trip, so workloads made of thousands of small files pay thousands of round trips. Amazon advertised a forty per cent reduction in write latency for files under 64 kilobytes in 2022, which tells you where the pain was, and their own performance guidance still recommends parallelism as the remedy. Compiling a large tree on a network mount is a lesson most people only need once.
The bill
These days you rent it by the hour, mostly without being told what you are renting.
Amazon EFS is NFS. Google Cloud Filestore is NFS and nothing else, since it does not speak SMB at all. Azure NetApp Files offers NFSv3 and NFSv4.1. And when a Kubernetes user asks for a volume with ReadWriteMany, which is the access mode you need whenever more than one pod writes to the same files, the driver underneath is almost always an NFS driver. The protocol from 1984 is holding up a large share of modern container storage, and a good number of the people relying on it have never opened an exports file in their lives.
The Kubernetes case is worth dwelling on, because it is where most engineers now meet the protocol without knowing it. The three access modes for a persistent volume are read-write on one node, read-only on many, and read-write on many. The first two can be served by a block device attached to a single machine, which is what the cloud platforms are good at and what they price cheaply. The third cannot, because two machines writing to the same block device at the same time produces a corrupted filesystem and a bad afternoon. So the moment a manifest asks for ReadWriteMany, something has to provide a real shared filesystem underneath, and the thing that provides it is nearly always NFS: either a managed service that speaks it, or a driver that mounts it, or an in-cluster server that runs it. A protocol designed for a room of workstations turns out to fit a cluster of nodes rather well, which is less of a coincidence than it looks, since the problem is identical and only the furniture changed.
The rate, in Amazon's Northern Virginia region: EFS Standard is 0.30 dollars per gigabyte-month. EBS gp3, block storage on the same platform, is 0.08. Roughly four times the price, for a filesystem protocol whose client and server together take up less space than a phone photograph.
Somewhere in your estate is a directory that three machines need to write to, and somebody is either paying rent on that directory or has spent a fortnight building it out of an object store and a lease table.
The comparison is not quite fair and it should be said plainly: the managed service buys replication across availability zones, capacity that grows without anybody resizing anything, and an operations team that is awake at three in the morning. Those are real things and some organisations should absolutely pay for them. What the invoice cannot buy is a different protocol. Small files are still small files, hard mounts still hang, and the identity model is the same one that ships in the base system.
The limit
The case for running it yourself has edges, and pretending otherwise would be silly.
AUTH_SYS is genuinely weak, and if the answer is Kerberos then somebody has to run a realm, which is a commitment rather than a command. NFS over a wide-area link is a bad idea regardless of who operates it. Object storage genuinely solves problems NFS never addressed, including access control per object, versioning, and serving the same bytes to a hundred thousand strangers. And a file server on your own hardware is a file server you have to back up, patch, monitor and replace, which is precisely the work people buy their way out of.
What the base system gives you is the choice, and the knowledge to make it. The protocol is there, documented, and costs nothing to try before deciding whether to rent it instead. An afternoon with zfs set sharenfs=on and a second machine will tell you more about whether you need the managed service than any comparison table, this one included.
The point
The reason NFS survived four decades of fashion is that it answered a question in the right place. Not in every application, once each and slightly differently, but underneath all of them, once.
An engineer at Sun in 1984 wanted to sit at any desk and find his own files. He got that, and forty-two years later so does a container scheduled onto whichever node had room, which is the same problem wearing a different hat. The bill is new. The protocol is not.