On 7 November 2024 Orion Newby, a freshman at Adelphi University on Long Island who has been diagnosed with Level 2 autism, handed in an essay he had written with the help of a tutor from the university's own support programme. His professor ran it through Turnitin, which returned an "AI-generated score of 100%". His parents ran the same essay through Grammarly's detector and through zeroGPT, and both reported "a 0% chance" that a machine had written it. That made two verdicts on one essay, and the university went with the program that accused him. On 28 January 2026 the Supreme Court of Nassau County found the violation "without valid basis and devoid of reason" and directed Adelphi to expunge it from his record.
The university's worry deserves to be taken seriously, because it is the right worry. An essay exists for the thinking that goes into it. A student who hands that thinking to a machine and copies out the answer learns nothing, and the degree at the end certifies a skill that never formed. Good use of these tools looks rather different: the writer thinks with the machine, argues with it, checks what it says and comes out knowing more than before. Teachers want to tell the two apart, and they are right to. The question is whether the instrument they bought can do it.
A model asked about a model
The first detectors measured surprise. A language model assigns every next word a probability, and a text the model finds predictable has low perplexity; GPTZero added burstiness, how much that predictability varies across a document, and the pair became the standard explanation in every newspaper. GPTZero itself has since moved on, and says so at the top of the very page that introduced the idea: "As of autumn 2023, GPTZero no longer uses perplexity and burstiness for its AI detection because we migrated to a deep-learning based architecture."
What sits underneath the others is easy to find, because their makers write it down. OpenAI described its own classifier, withdrawn in July 2023, as "a language model fine-tuned on a dataset of pairs of human-written text and AI-written text". Turnitin's white paper of August 2024 calls its engine a "transformer classifier" whose output is "a single real number between 0 and 1", and reports that it had processed more than 250 million papers by June 2024. Pangram describes itself as "a transformer-based neural network". Binoculars, the best-regarded method in the research literature, compares the text against two open Falcon models at once. In the Stanford study of 2023, most of the seven commercial detectors tested ran on GPT-2 as their "underlying backbone model". Every one of them is a language model, or a pair of them, reading an essay and reporting how much it resembles the work of a language model.
The student, meanwhile, was not allowed to use one.
There is an older version of this arrangement. In 2004 Luis von Ahn and his colleagues described the CAPTCHA as "a program that can generate and grade tests that it itself cannot pass (much like some professors)". The detector turns the joke round: it grades a test that its own relatives pass without effort, and hopes nobody notices the family resemblance.
What a model finds predictable
Measuring resemblance to a machine has a mechanical consequence, and the Stanford team led by Weixin Liang found it in 2023. They gave seven detectors 91 essays written by people taking the TOEFL, the standard English test for non-native speakers, all collected before ChatGPT existed. The detectors marked an average of 61.22 per cent of them as machine-written, and 89 of the 91 were flagged by at least one. Then the team ran the experiment backwards. They took essays by American eighth-graders, which the detectors had judged almost perfectly, and asked a model to simplify the word choice. The false positive rate rose from 5.19 per cent to 56.65.
A model finds plain, regular English predictable, so plain, regular English looks like a model. That catches the second-language writer, and it catches anybody whose sentences come out tidy by nature or by training. The authors' advice was unambiguous: "we strongly caution against the use of GPT detectors in evaluative or educational settings."
What one per cent costs
Turnitin announced its detector in March 2023 with "a less than 1% false positive rate" and launched it in April. In May its chief product officer wrote that "real-world use is yielding different results from our lab". Below 20 per cent of detected machine text, false positives turned out more frequent ("This is inconsistent behavior", in the company's own words), the rate for individual highlighted sentences stood at about 4 per cent, and the minimum length for a document to be checked went up from 150 words to 300. Calling that a measurement is a bit rich when the vendor cannot yet say why it behaves as it does.
Vanderbilt University did the arithmetic in August 2023. "Vanderbilt submitted 75,000 papers to Turnitin in 2022. If this AI detection tool was available then, around 750 student papers could have been incorrectly labeled as having some of it written by AI." Vanderbilt switched the detector off, noting that Turnitin "gives no detailed information as to how it determines if a piece of writing is AI-generated or not".
At Australian Catholic University the arithmetic arrived in person. In 2024 the university used the detector to accuse about 6,000 students of misconduct; "around one-quarter of all referrals were dismissed following investigation", and any case in which Turnitin was the only evidence was dropped at once. ACU abandoned the tool in March 2025. An independent test by Debora Weber-Wulff and colleagues across 14 tools put the false positive rate anywhere between 0 per cent for Turnitin and 50 per cent for GPTZero on their test set, and concluded that the reports "cannot be used as the only basis for reporting students for cheating".
One per cent sounds like precision. Multiplied by an intake, it is several hundred students a year at a single university, each of whom has to prove a negative to somebody holding a number.
The strongest case for the tools
The tools have improved, and the best evidence for them deserves its full weight. In September 2025 Brian Jabarian and Alex Imas tested three commercial detectors for the National Bureau of Economic Research on 1,992 human passages across six everyday genres, from novels to restaurant reviews to CVs. Pangram stayed below a false positive rate of 0.005 at every threshold they tried, and at the stricter settings recorded none at all. Binoculars reports more than 90 per cent of machine text caught at a false positive rate of 0.01 per cent, so on the right material detection plainly works.
The conditions are the whole story. They come down to long passages of unedited output in a genre the detector has been trained on, and the same NBER paper records Pangram's error rate rising on shorter passages. Change the text even a little and the figures move fast. A paraphrasing model called DIPPER dropped DetectGPT's detection rate from 70.3 per cent to 4.6 at a fixed false positive rate of 1 per cent. In RAID, a benchmark of more than six million generated texts, swapping a few Latin letters for Cyrillic ones that look identical took Originality from 85.0 per cent to 9.3. Liang's team had their machine-written college essays rewritten by the same model with a request for "literary language", and detection fell from as much as 100 per cent to at most 13. Binoculars' authors say plainly that they "do not guarantee any performance in settings where a motivated adversary tries to fool our system", and OpenAI's own announcement already conceded in 2023 that "it is unclear whether detection has an advantage in the long-term".
Put the two halves together and the result is uncomfortable. The student who means to cheat runs the essay through a paraphraser and passes. The student who wrote plainly, or in a second language, or very tidily, has no idea there is anything to evade, and that is who the detector flags.
Why recognising still matters
None of this makes recognition pointless. Knowing whether a text came from a person matters for a handful of plain reasons: who stands behind a claim, whether anybody checked it, what the writer can actually do, and whom to ask when it turns out to be wrong. A teacher needs to know what the student can manage alone, and an editor needs to know who signed the piece. The skill is worth having, and the research has found out where it lives.
Jenna Russell and her co-authors tested it on people, in a paper for ACL 2025. They gave 300 English non-fiction articles to five annotators who use ChatGPT a great deal for their own writing and had no specialised training. Taken by majority vote, the five misjudged one article in 300: 99.3 per cent correct, with no human text wrongly accused. On articles that had been deliberately "humanised" to slip past software, the panel still caught every one, while Binoculars at its strict setting caught none. Pangram, the strongest of the tools on the same set, reached 98.0 per cent with 2 per cent false positives. And people who rarely use these tools, given the same task, caught 56.7 per cent of the machine texts whilst wrongly flagging 51.7 per cent of the human ones, which is a coin toss with paperwork.
The readers who did best had learnt the machine's habits by working with it, the over-tidy paragraph, the vocabulary that never quite commits, the list that always has three items and the conclusion that arrives on schedule. Recognition comes with good use, in other words. The NBER authors object that such experts are "not a scalable solution", and they have a point about the pile. Every accusation, though, ends up in front of a person anyway, at a hearing or, as at Adelphi, in court, and every serious study of the tools says the same thing about the score, that it can raise a question and cannot answer one.
The limit
On the right text the best tools are good. A long, unedited submission in a familiar genre, run through a detector of Pangram's class, comes close to the laboratory figure. There a high score is a reasonable reason to sit down with the student and talk about the essay.
Reading does not scale to 250 million papers. A score may sort the pile and decide which essays a person reads first. The decision about a student belongs to the person, after a conversation, and Weber-Wulff's team say so in as many words.
One reader is not enough. Individual annotators in the ACL study were wrong far more often than the majority of five, and on humanised texts one of them wrongly accused nearly a quarter of the human articles. A small panel of readers who know the tools is the instrument; the ACL study used five, and it was their majority that came out at 99.3 per cent.
Watching the process beats judging the product. Drafts and an oral defence of the essay tell a teacher more than any score, and they cost time that a detector licence was bought to save. That is cold comfort for a budget, and still rather cheaper than a hearing in Nassau County.
Any student may fairly ask the professor to prove it without a machine, and the professor who can usually turns out to have used one.