Vivian Voss

The Code You Did Not Write

philosophy ai liability ownership

A fortnight ago this section closed a series at the silicon, the layer with the fewest words and the most consequence, and it closed on a particular word: that one should not let the term sovereignty carry, in the present tense, a weight that only the next decade's foundries can bear. Last Sunday was an aside, on knowing when a thing is finished. This one picks that earlier word up, one layer higher, where most of us now work every day and have quietly stopped reading.

The scene has become a genre. It is three in the morning, a service is doing the wrong thing, and someone has the offending module open. They run git blame. A name appears, and a date, and a commit message. The name belongs to a colleague who is awake and on the call, and who looks at the line and admits, honestly, that they do not know why it is the way it is. They did not write it. They accepted it. It compiled, it passed the tests, it read as plausible at the speed a review runs, and it went in under their name. The code is deployed, it is theirs in every sense the law and the org chart recognise, and it is authored by no one in the room.

This is the ordinary case now, not the edge case. And it raises a question the tooling has made it easy not to ask: when the answer arrives faster than the understanding, who wrote the code, who can read it, and who answers for it when it fails?

The Author Who Only Accepted

Authorship used to be a fairly solid thing. A line of code existed because a person held a model of the problem in their head, decided what the line should do, and typed it, and could therefore tell you, months later, what they had been thinking. Run against a file, git blame was a rough but honest instrument: it pointed at the person who had made the decision. Its authority rested on a quiet assumption, that the last hand to touch a line was the mind that had shaped it.

That assumption has come loose. A generated line is produced by a model that held no model of your problem, drew on patterns from code it was trained on, and proposed something that fit the shape of the request. A person then accepted it. Running git blame now points at the acceptor, not the author, and it cannot tell the difference, because at the level of the commit there is no difference to see. The instrument still runs. It has simply stopped meaning what it used to mean.

The gap this opens is not merely anecdotal. In its 2026 survey of engineering practice, the security firm Aikido reported that only twenty-one per cent of organisations believe artificial intelligence will ever write secure code without human oversight. Read plainly, the field does not think the tool can be left unattended, which makes the human review the load-bearing step, and its quiet thinning-out under deadline the interesting part. That is not a moral failing of individual engineers, who are under the same delivery pressure they always were, only now with a faster way to produce the appearance of progress. It is a structural fact about what the commit has come to certify. A signature that means only “I did not object” is not, on reflection, a signature at all.

The field's own verdict on AI-generated code (Aikido 2026) 21% believe AI will ever write secure code without human oversight 69% have already found flaws introduced by AI-generated code 1 in 5 has had a serious incident traced to AI-generated code The profession knows it is signing things it has not read.

The Line You Cannot Draw

There is a deeper problem underneath the acceptance, and it is the one this series is really about. You can read a generated line, in the narrow sense that the characters are on your screen and you understand what they do. You cannot read where it came from. You do not know which training examples shaped it, whether it reproduces a pattern from code under a licence you would never have chosen, or what it silently declined to handle. The model that produced it cannot be forked, cannot be inspected in any way that would answer these questions, and cannot be called to account. It is available to you, and it is not legible to you, and those are different words.

The industry has noticed, and its response is instructive. The current proposal is the AI bill of materials: tag every component with the model that generated it, keep the prompt logs, record the provenance in commit metadata, so that one day a line can be traced to its origin. It is a serious and sensible effort, and it is also an attempt to draw, after the fact, a line the architecture never drew in the first place. It records that a line came from a model. It does not make the model readable, and it does not make the line accountable, because provenance you bolt on afterwards is a label, not a boundary.

What the blame annotation names Then — the author a person held the problem in mind and decided what the line should do the last hand was the shaping mind Now — the acceptor a model held no model of the problem a person pressed accept the last hand only did not object The instrument still runs. It has stopped meaning what it used to mean.

This is the point at which openness stops helping. An open-weights model, downloaded and run on your own hardware, is more sovereign than an API you rent, and it still does not tell you what it learned. Generated is not authored; available is not legible; the code you can run is not the code you can read. The series before this one spent three Sundays separating ownership from openness at the licence, the architecture and the silicon. This is the same separation, one layer up, and the layer is made of language rather than metal. Aikido's same survey found that sixty-nine per cent of organisations had already found vulnerabilities introduced by AI-generated code, and that one in five had suffered a serious security incident traced to it. The line one cannot draw is not a theoretical concern. It is a fifth of the field, already.

Who the Law Finds

While engineers have been debating provenance, the law has quietly settled the accountability question, and it has settled it in the direction the org chart already pointed. It has done so in two jurisdictions that agree on very little, which is what makes the agreement worth noticing.

In California, a 2025 statute, AB 316, removed a defence before anyone had finished making it. In an action alleging that an artificial-intelligence system caused harm, a defendant who developed, modified, or used that system may not argue that the AI acted autonomously and that the harm is therefore not theirs. The law does not impose strict liability and does not need to. It simply closes the exit marked “the machine did it,” and leaves the ordinary framework of fault standing, now with the human firmly inside it.

In the European Union, the revised Product Liability Directive, adopted in 2024 and applying from December 2026, does something structurally similar from the other end. It writes software, firmware, and AI systems explicitly into the definition of a product, under the same no-fault liability regime that has long covered a faulty kettle. Where a claimant would struggle to prove a defect in something as opaque as a model-shaped codebase, the Directive shifts the burden: if the producer cannot show that it followed the required processes, the court may presume the defect. Failing to review becomes, in effect, evidence against you. The companion Cyber Resilience Act supplies the processes whose absence the court will notice.

Two jurisdictions, reasoning apart, one principle California — AB 316 (2025) no “the AI acted autonomously” defence ordinary fault stands, human inside it not strict liability EU — PLD 2024/2853 (Dec 2026) software and AI are a product no-fault liability, burden shifts unreviewed becomes evidence against you the entity that ships it owns it whoever, or whatever, first wrote it Review is becoming a duty the law assumes you performed.

Two legal systems, reasoning independently, arriving at the same place: the entity that ships the code owns the code, whoever, or whatever, first wrote it. The principle is not specific to either. The same reasoning runs in Delhi and in São Paulo; which jurisdiction codifies it first is a matter of legislative calendar, not of principle. Review, which used to be a courtesy the schedule often skipped, is on its way to being a duty the law assumes you performed.

The Limit

It would be dishonest to leave this sounding like a case against the tool, and it is not one. These models are a genuine gain. They cross the blank page, they lower the cost of a first attempt, they put a competent draft in reach of people who would otherwise have had none, and treating that as nothing would be its own kind of dishonesty. The argument here is narrower, and it survives all of the gain: acceptance is not authorship, and accountability does not transfer to the thing you accepted from.

Two honest concessions. A human can write code no one can maintain; inherited legacy is full of lines whose author left years ago and whose logic no one now carries in their head, and we did not need a model to arrive there. What is new is not the existence of unreadable code but its rate and its plausibility, code produced faster than it can be understood and polished enough that no one feels the need. And the counter-image is not a fantasy: on a coherent base system, the kind one team writes and keeps, a line has a known author, a named maintainer, and a delivery you can verify, which is the opposite of a weight no one can read. But even that only draws the provenance line. It does not do the reading for you. Understanding is a practice, not a possession, and no boundary, however cleanly drawn, understands the code on your behalf.

The Point

You can import the code. You cannot import the understanding that made you independent of the thing you imported it from, and you cannot delegate, to a model that holds no model of your problem, the accountability that arrives with your name on the commit. The tool writes the line. It does not attend the three-in-the-morning call, it does not appear in the Directive's definition of a producer, and it will not be the one explaining, to a room or to a court, why the code did what it did.

The licence, in the series before this one, was the layer with the most words. The code you did not write is the layer with the most speed, and the least memory.

Next Sunday: who governs the thing you cannot fork?