There is a table on the desk. Twenty columns, twenty rows, a five-digit figure in every cell. It flutters in six times a day, that is what one is ultimately paid for, and the only job is to say whether any of the four hundred figures is off. Not which one. Only whether anything looks out of place.
On the first day that goes rather well. On the third it still does. What matters is the fourteenth, the day with something going on at home, or simply the day when the eye slides off a pattern it has seen eighty-four times already.
None of this is about diligence, and it has been measured properly: for seventy-eight years now, always with the same result.
The Deficit Is Older Than the Computer
Norman Mackworth sat observers in front of a clock in 1948. The hand advanced in even steps and now and then made a jump twice the usual size. Two hours of watching, report every big jump, nothing else on the sheet. Detection fell by ten to fifteen per cent within the first half hour, and the observers knew the jumps were coming, which rather makes it worse. He called it the vigilance decrement, and in the seventy-eight years since, nobody has found the training that undoes it.
Jeremy Wolfe put the more awkward question in Nature in 2005. Same images, same trained observers, only the frequency of the thing being hunted changed. Common targets were missed about seven per cent of the time. Rare ones went past at thirty per cent and beyond, and the effect held when observers were briefed about it in advance. Radiology and airport screening report their own figures in the same direction, and those people are not amateurs.
The opposite failure is documented just as thoroughly. Somewhere between 85 and 99 per cent of clinical monitor alarms are false or not actionable, so nurses stop responding to them, and the Joint Commission has been writing sentinel alerts about it since 2013. There are two reliable ways to lose an observer: give them something to find far too seldom, or something to react to far too often.
Which gives us the thing the current debate tends to skip: anyone who wants a tool at this point, something to take the job off them, is entirely right. That is no laziness. It is the sensible response to a well-measured limit of one’s own construction.
And the tools we built for it over the centuries all worked the same way. They did not make the human more attentive; they gave the error a shape, so that it gave itself away. Double-entry bookkeeping, as Pacioli described it in 1494, writes every figure down twice, and a transposed digit becomes visible at all only because the books stop balancing. The check digit in an IBAN does the same for an account number, the checksum for a freight note, the second signature for a payment order. Each of these adds redundancy so that the mistake announces itself before anyone has to go looking.
That is the standard against which whatever moves into the same seat has to be measured.
The Copier That Lied Politely
In the summer of 2013 David Kriesel copied some building plans, and the printout came back with different room areas than the original. Cleanly set, correct typeface, no ragged edge, no artefact, not the faintest hint that anything at all had happened.
The compression was to blame, in a way that had nothing to do with text recognition. JBIG2 hunts through a scan for image regions that resemble one another, stores one of them and drops it back in wherever the pattern matcher reports a match. Let that matcher work a shade too coarsely and a region lands somewhere it merely looks similar. A six became an eight this way, and the eight was then not a damaged six either. It was a perfectly good eight.
Xerox eventually published the list itself, and it is not a footnote: four product families and dozens of model lines, from the ColorQube 87XX through most of the WorkCentre range and on to the BookMark. These are ordinary office machines, and they stand where documents are turned into records, which is to say in planning departments, architects’ practices, tax offices, hospitals and engineering firms worldwide. The company sold through resellers, so no customer list could be assembled, and nobody could say then, or can say now, how many of those machines took the patch and how many carried on as before. The reported cases run from the structural calculation for a bridge through fuel figures to medication doses.
Germany’s federal security office drew its conclusion in March 2015 and wrote it into the guideline that governs scanning for legal record-keeping: any compression working by symbol coding, JBIG2 first among them, is unsuitable. The term of art in that guideline is ersetzendes Scannen, replacement scanning, and it describes the practice of scanning a document and then destroying the paper. That is the part worth holding still for. A process that alters a figure without leaving a mark had been cleared for the one workflow that removes the only copy anybody could have checked it against.
Here is the point that matters, and it is of a fairly general nature. The error arose from the removal of redundancy and inherited the shape of the correct thing as it did so. A checker could not have done anything wrong, because there was nothing wrong to see on the page. There was a number, like all the others.
A language model works one floor up on the same principle. It produces the most probable continuation, and the most probable continuation is by construction the output that looks least like an error. The mistake is therefore not scattered at random across the page: it is optimised for going unnoticed, and it arrives with reasoning that sounds coherent, because coherence is what the process was trained on.
Ten Questions Put to a Logfile
Which is a claim, and this column measures claims.
The material is the SSH log from the public Loghub corpus, 655,146 lines of a real authentication log, so that any reader can recompute every figure. Ten questions of the sort asked on an ordinary morning: how many failed logins, from which networks, which username was tried most often, which day was worst. Four runs over the same data.
The first surprise arrived early. The model got the figures right. Every one of them, on a slice small enough to fit in its context, without a single tool. Anyone expecting a hallucinated number here can stop hoping.
The second surprise was worse for the author than for the model. Asked which address produced the most failed logins, it answered with a tie: two addresses on thirty-five each. My awk one-liner had reported one of them, because it ended in head -1 and a tie does not survive that. Asked how many distinct usernames had been tried, it said twenty-nine where my script said twenty-eight, and it was right again: one of the names carries a leading space, which my pattern quietly stepped over. Two errors in the measurement, both mine, both found by the thing under test.
So much for the model getting things wrong. The interesting part is what happens when the same question is asked more than once.
Three agent runs over the full log, same file, same ten questions, one after another. The number of distinct usernames came back as 1,719, then 1,720, then 1,721, depending on how each run decided to treat an empty name that appears thirteen times. The busiest day came back as 3 January when the run counted lines, and as 4 January when it counted attempts, because a message repeated 6 times line is one line and six attempts. Each of those answers is reasoned, defensible and correct under the definition its own run chose. No two of them are the same.
That is the finding, and it is sharper than the one I went looking for. The model does not answer the question wrongly. It answers a slightly different question each time, supplies the definition along with the answer, and nothing in the output marks the moment the definition changed. The awk line has no such courtesy. It forces the person asking to decide up front whether a repeated message counts once or six times, and it then makes that decision visible, wrong or right, to anyone who reads it. My twenty-eight was wrong in public and is now gone for good. The model’s twenty-eight turned up in one run out of five and will turn up again.
Eight runs came to 5.18 dollars all told, and that figure carries the harness the runs went through, not the bare model call. The awk line cost nothing, and it is the only figure in this protocol that will still be the same figure tomorrow.
Two Curves That Cross
What a person can check has been quantified for some time. SmartBear measured two and a half thousand code reviews across 3.2 million lines at Cisco and put the useful limit at two to four hundred lines in one sitting; beyond roughly five hundred lines an hour, defect detection falls away, and after sixty to ninety minutes at the screen it does the same. Average attention on a screen ran to about two and a half minutes in 2004 and is put at roughly forty-seven seconds for the 2020s, which serves here as a boundary condition and not as proof.
That curve is flat. It was flat in 1948 and it is flat today.
The other one is not flat in the least. LinearB’s 2026 benchmark covers some 8.1 million pull requests from 4,800 organisations: teams leaning hard on AI merge around 98 per cent more changes, review time per change climbs steeply, and a growing share goes in with no review recorded at all, which nobody decided as such, there is simply nobody left keeping up. GitClear measures duplicated blocks over the same period at a record 73 per million changed lines, whilst moved code, the sort that indicates an actual refactor, has slid from 24.1 per cent in 2020 to 3.8 per cent in the 2026 edition. In accounting the same movement is called the touchless rate, and the better houses now report eighty to ninety-five per cent of documents passing through without human contact.
The end state is described by that, and it works as a balance sheet rather than a horror story. Machine speed at the top of the chute. At the bottom sits the checker from the first section, with four hundred lines in a sitting and forty-seven seconds of attention, and he is now expected to catch a class of error built specifically not to look like one.
A tool that is right often enough is a tool nobody is checking.
Then there is the stage that appears in no procedure document. Late in the afternoon a person knows perfectly well that they stopped reading a while ago, and waves it through anyway, because the meeting is at four and this really is the last one. I would not call that a character flaw. It looks more like a process that has quietly moved its guarantee onto the one component never rated for continuous duty.
What Follows
The comfortable answer is to take the tool away again, and it holds up right until you follow it through. It puts the checker back in front of the table on his own, and we have known since 1948 how he does there. It also confuses the reliability of a human with the jurisdiction of one. Anyone who rails against the models today and compares four hundred figures by hand tomorrow has chosen the worse of two error sources, and will quite often think of himself as the thorough one in the room.
The workable answer is the old one anyway. You restore the redundancy, by machine, because at this volume it cannot be restored by hand. Where the answer has to be exact, it belongs to a tool that says the same thing twice and whose working anyone can retrace. Where the question is genuinely open, where nobody knows the format and nobody knows what happened at 03:14, the model beats the alternatives, and its answer then goes through the tool before it goes anywhere near money or data.
Xerox shipped a patch eventually. Nobody could say which machines received it, the documents scanned in the years before were long since filed, and wherever the paper had been destroyed on purpose there was nothing left to check them against.