How AI Detectors Actually Work: Understanding Perplexity, Burstiness, and Stylometry

What To Know First

AI detectors don’t “read” your writing. They don’t understand meaning, tone, or whether you plagiarized anything. Instead, they’re mathematical pattern-matching engines that analyze your text’s statistical fingerprints—and if those fingerprints look machine-smooth, they flag it.

In 2026, most commercial AI detectors rely on five core signals: perplexity (how predictable each word is), burstiness (how varied your sentence rhythm is), token probability (whether words follow the most obvious path), stylometry (your unique linguistic fingerprint), and a trained machine-learning classifier that weighs all these signals together.

The output is always a probability—not a verdict. A score of “82% AI” is an estimate of confidence, not proof of academic dishonesty. Understanding exactly how these signals work, what they measure, and why they sometimes make mistakes is the single most useful thing you can do to know whether a detector score actually means anything.

What Is Perplexity and How Detectors Use It

Perplexity measures how “surprised” a language model is by your words. It asks: given the context, how predictable is the next word?

Here’s a concrete example:

• AI version (low perplexity): “Climate change is a global problem that requires urgent attention.”
  Every word is exactly what a language model would most likely predict given the context.
• Human version (high perplexity): “Honestly, it feels like the summers just keep creeping hotter—and we keep pretending we’ll deal with it next year.”
  A language model would find this quirky phrasing, casual tone, and unexpected combination of ideas.

Mathematically, perplexity is calculated as the exponentiated average negative log-likelihood of a sequence. If we look at a text of N words and let P be the probability of each word given all the words before it, the perplexity score is:

PPL = exp(−1/N × Σ log P(wi))

The lower the perplexity, the more predictable—and thus more machine-like—the text appears. AI language models generate by selecting from a narrow probability distribution, so their output tends toward low-perplexity, smooth prose. Human writers, by contrast, have access to far more vocabulary, slang, emotional expression, and contextual detours. When a human text shows very low perplexity across the board, it may signal unusual formulaic writing—which is rare and worth examining.

Important caveat: Famous, widely-reproduced texts (the Declaration of Independence, Wikipedia articles, classic literature) are memorized during LLM training. When a detector runs them through its model, they score low in perplexity—and have been falsely flagged as AI-generated. This is called the Pangram Labs paradox.

What Is Burstiness and Why It Matters

Burstiness measures the variation in your sentence length and rhythm. Humans naturally alternate between short, punchy sentences and long, complex ones. AI text tends toward uniform sentence length.

Here’s the same idea, written two ways:

• Human (high burstiness): “The results were significant. But not as expected. What does this mean? It means we need to reconsider everything.”
  Sentence lengths: 10, 8, 9, 13 words. Highly varied rhythm.
• AI (low burstiness): “The results were significant and indicated meaningful patterns. Furthermore, these findings were contrary to initial expectations. This discrepancy suggests further investigation is necessary.”
  Sentence lengths: 15, 15, 17 words. Uniform and monotonous.

Detectors calculate burstiness as the standard deviation (or coefficient of variation) of sentence lengths across the document. A high standard deviation means high burstiness—your writing has natural peaks and valleys. Low burstiness means every sentence feels the same length and cadence.

Research by OpenAI and independent researchers has shown that combining perplexity and burstiness scores significantly improves detection accuracy over using perplexity alone. When both signals are low, the probability of AI generation rises sharply.

Token Probability: The Building Blocks of Detection

To a computer, text isn’t made of words—it’s made of tokens. Tokens can be whole words, fragments of words, or even individual characters. Language models generate text by predicting the next token in a sequence, like an endless game of “finish the sentence.”

Detectors zoom in on these token-level probabilities. If the sequence is too regular, too smooth, or too mathematically likely at every step, the detector grows suspicious.

Here’s why this matters: a 2023 study by Jozzo et al. found that humans are “consistently worse than relatively small language models like GPT-Neo-1.3B at next-token prediction.” In other words, AI models are better than humans at predicting human text. This undermines the idea of “unpredictability” as a reliable human signature—because machine models already understand how humans write better than humans themselves do.

Stylometry: Your Linguistic Fingerprint

Stylometry is the science of analyzing a writer’s unique linguistic fingerprint—the measurable habits that make prose distinctive. It looks past content entirely and examines patterns:

Two 2025 peer-reviewed studies confirmed stylometric methods effectively distinguish human from LLM-generated text:

• Przystalski (Expert Systems, 2025) found stylometric features achieve strong classification accuracy across genres.
• O’Sullivan et al. (Nature Scientific Reports, 2025) confirmed effectiveness in creative writing contexts.

Machine Learning Classifiers: AI Catching AI

Here’s the irony: most modern AI detectors are themselves AI. They’re trained on huge datasets of labeled human and machine-generated text, learning the statistical boundary between the two.

When you submit new text, the classifier compares its signals against everything it has seen and outputs a probability. It’s like training a sniffer dog—once it learns the difference between the scent of chicken and beef, it can tell you which one is in your bag. The detector “sniffs out” AI writing by matching new text against its training experience.

Common algorithms used include:

These classifiers learn features automatically during training—often including perplexity scores, sentence-length distributions, punctuation frequency, text entropy, and syntactic complexity. When trained, they don’t compute each signal manually; they learn which combinations of patterns matter most.

How the Signals Combine Into a Score

No single signal decides the verdict. A detector blends all five inputs—perplexity, burstiness, token probability, stylometry, and classifier match—much like a doctor reading several test results before reaching a diagnosis.

Here’s the full pipeline:

1. Input text is tokenized using a consistent tokenizer (e.g., GPT-2’s byte-pair encoding)
2. Log-probability scores are computed for each token by a reference model
3. Perplexity is calculated as the exponential of mean log-probability
4. Text is segmented into sentences; standard deviation of perplexity scores approximates burstiness
5. Stylometric features (vocabulary, punctuation, function words) are extracted
6. All signals feed into the trained classifier
7. Output: a probability score (0–100%) and sometimes a soft label (“likely AI”) or “likely human”

The output is never a certainty. A score of “82% AI” means the model’s confidence based on patterns it has learned—it is not proof, and no responsible institution treats it as such.

The Accuracy Problem: What Research Actually Shows

AI detectors are probabilistic, not definitive. The research is clear:

Key takeaway: GPTZero claimed a 99% accuracy rate. Independent research consistently found ~52%. Turnitin claims 98%+ accuracy; independent studies find inconsistencies, particularly with heavily-edited drafts and non-native speakers.

No tool can provide complete accuracy. Even the most confident providers admit their tools cannot be used as definitive evidence.

Why Detectors Sometimes Get It Wrong

Understanding the mechanism explains the blind spots:

1. ESL and Non-Native Speakers

Writing that is clear, simple, or formulaic—including text by many non-native English speakers—produces low-perplexity, low-burstiness patterns that detectors may misread. A Stanford study of 91 TOEFL essays found a 61.3% false positive rate. The detector is matching a pattern, not reading intent.

2. Short Texts Carry Too Little Signal

A handful of sentences rarely gives enough statistical variation to judge confidently. Papers under 250 words score “wildly fluctuate” across runs. A 300-word reflection scoring “71% AI” is borderline evidence at best.

3. Formulaic or Structured Writing

Lab reports, legal documents, and technical documentation naturally exhibit low burstiness. A sales email template used repeatedly will have low perplexity and low burstiness. This is innocent human writing that happens to match machine patterns.

4. Edited or Paraphrased AI Text

When users employ basic paraphrasing tools (like QuillBot) or manually rewrite AI output, detection accuracy drops dramatically—falling between 40% and 60% in some studies. Both platforms see a massive accuracy drop when texts are edited.

5. Adversarial Editing

Skilled AI operators who edit output rigorously can raise perplexity and burstiness to human levels. Detectors trained only on raw LLM output have never seen examples of AI text that has been manually rewritten five times. The detector’s training set is finite; humanization techniques exploit that gap.

6. Model Drift

Detectors are tuned to the language models that exist today. As new models change their token patterns, yesterday’s detector becomes less reliable. A detector trained on GPT-3 output may perform poorly on Claude or Gemini, which have different sampling behaviors.

What Your Score Actually Means

A detector score is a pattern-match estimate. It answers one question: “Does this text look statistically like what I’ve seen from an LLM?” It does not answer: “Did this person cheat?” “Was this written entirely by AI?” or “Is this academically dishonest?”

A responsible process uses the score only as a prompt to look at:

• Draft history and version control
• Author’s ability to discuss the content
• Contextual factors (non-native speaker, short text, formulaic genre)
• Multiple tools’ agreement (not a single tool’s verdict)

The correct response to an ambiguous flag is conversation—not accusation.

How to Lower Your Detection Risk (Responsibly)

This guide exists to explain how detectors work, not to teach evasion. But understanding the signals helps you write authentically:

What actually works:

• Use AI as a research assistant, not a ghostwriter. Brainstorm, outline, and get source ideas—but write the final text yourself.
• Add specific examples, names, and numbers you verify independently.
• Vary sentence length deliberately. Mix short fragments with longer, complex sentences.
• Inject your natural voice. Even if it’s less “polished” than AI, it’s more defensible and more human.
• Keep draft files showing your writing evolution. Version histories, notes, and outlines are your strongest defense.

What doesn’t work:

• Inserting random typos or errors. This raises perplexity artificially, but it also makes your writing worse.
• Using “humanizer” tools without understanding. They add fillers, redundant phrasing, and colloquialisms that hurt quality.
• Assuming one score is definitive. No single detector score should be treated as proof.

Why This Matters More Than Ever

In 2026, AI detectors are widely deployed in education, publishing, hiring, and content moderation. Students and writers face real consequences from algorithmic flags. Understanding the technology behind detection—what it measures, how it calculates, where it fails—is the foundation of informed participation in an AI-aware world.

The core truth: AI detectors are probability classifiers, not truth machines. They analyze patterns, not honesty. They measure predictability, not integrity. A low score doesn’t prove anything about what you actually did. And a high score doesn’t prove misconduct either.

Related Guides

• How to Write Original Content That Avoids AI Detection — Practical strategies for authentic academic writing
• AI Detection Accuracy: Understanding False Positives — Why detectors flag human writing and what you can do
• How to Appeal an AI Detection False Positive — Step-by-step student guide
• Ethical AI Writing Tools for Students — When and how to use AI ethically

Ready to Check Your Work?

If you want to see how your writing scores on AI detectors, scan it with Paper-Checker’s free AI detection tool to compare against multiple engines before submission. Our tool runs detailed analysis and provides transparent reports highlighting AI probability scores and which signals were flagged.

Check your work with Paper-Checker