Can You Tell AI Writing from Human? Most People Can't

Every week, another AI detection tool launches with claims of 99% accuracy. Teachers use them to flag student essays. Editors use them to screen freelance submissions. Publishers use them to gatekeep content. And yet, the academic literature on the actual performance of these tools paints a much more complicated picture.

Understanding why detection is hard — and likely to stay hard — matters for anyone writing, editing, publishing, or consuming content in 2026.

How Detection Tools Work

AI text detectors are themselves machine learning models trained to distinguish AI-generated text from human-written text. They look for statistical patterns: the distribution of word choices, sentence length variation, perplexity (how surprising the word choices are given context), and burstiness (variation in sentence complexity).

Human writing tends to be higher-perplexity — more surprising, less predictable, with greater variation in rhythm and structure. AI writing tends to be lower-perplexity — statistically more "average," more predictable, more consistent.

This is the theoretical basis. The practical performance is where things fall apart.

The False Positive Problem

The most significant documented problem with AI detection tools is false positives — flagging human-written text as AI-generated. Research published in 2023 and 2024 found that non-native English speakers are disproportionately flagged as AI writers. The reason: non-native speakers often write in the same lower-perplexity, more-predictable patterns that AI models exhibit.

In one widely-cited study, GPTZero flagged the US Constitution as AI-generated. Another study found that academic writing — which is intentionally precise and formulaic — is routinely misidentified. The tools are, in effect, detecting "writes like a machine" rather than "was written by a machine," and those are not the same thing.

The Arms Race Dynamic

Detection models are trained on text from current AI systems. But AI models evolve rapidly. A detection model trained predominantly on GPT-3.5 outputs will struggle with GPT-4 or Claude outputs, which exhibit different statistical signatures.

There's also a direct adversarial dynamic: the moment a detection pattern becomes known, AI models can be prompted to avoid it. "Write this in a way that varies sentence length" and "add some intentional imperfections" are prompts that substantially reduce detection rates on most current tools.

This isn't a solvable problem through better models — it's structurally an adversarial race that detectors cannot win permanently.

What the Research Actually Says

A 2024 meta-analysis of AI detection tools across multiple academic contexts found average accuracy rates in the range of 70-80% when tested on out-of-distribution text — text that differs from the training distribution in style, domain, or origin model. Under ideal conditions (testing on text similar to training data), some tools approach 90%+.

The 90%+ headline accuracy figures that tool vendors publish are typically measured on their own test sets — which are optimized for their model's strengths. Independent evaluation consistently shows lower performance, particularly for:

• Short texts (under 250 words)
• Non-native English writing
• Technical or highly specialized domains
• AI-assisted (rather than fully AI-generated) content
• Text that has been lightly edited after generation

What Humans Can Actually Detect

Human detection is not better than tools — multiple studies have found humans operating at 50-55% accuracy when presented with AI text mixed with human text. We are slightly above chance, which is not a meaningful capability.

What humans are good at detecting is incoherence, factual errors, and a certain vacuous quality where the text sounds authoritative but on closer reading says nothing specific. These are real tells — but they reflect low-quality AI output more than AI output generally. Well- prompted, well-edited AI text routinely fools human reviewers.

The Practical Upshot

For writers: detection tools are unreliable enough that false positive accusations are a real risk. Maintaining records of your drafts and writing process is worth doing not because you need to prove innocence but because it's good practice.

For editors and publishers: AI detection scores should not be used as sole grounds for rejection. They should be one signal among many, weighted by the consequences of being wrong.

For readers: the question of whether text was written by a human or AI may matter less than whether it's accurate, thoughtful, and useful. Those qualities are harder to fake — and harder to detect with any tool.

Free Game

AI or Human? Test Your Detection Instincts

A daily puzzle with real samples. Can you tell AI writing from human? See how you score against thousands of other players.

More articles