Perplexity.

Perplexity

Perplexity measures how predictable a passage is: if every next word is exactly what a language model expected, perplexity is low. If the text frequently surprises the model, perplexity is high. AI-generated text tends to have lower perplexity than human text because it's drawn from the same distributions the reference model is scoring against.

Perplexity alone is a weak signal, careful academic writing also has low perplexity. Modern detectors combine perplexity with burstiness and a learned classifier.

A worked example

Take the sentence "The mitochondria is the powerhouse of the cell." A reference language model expects every word, perplexity is very low. Now take "Look, the cell's tiny engines, those mitochondria, they're not just sitting around." Same factual content, but now the reference model is repeatedly surprised by the word choices. Perplexity is much higher. Human writers produce the second pattern far more often than untouched LLM output does.

Where perplexity fails

Perplexity is a weak signal on careful, formal academic prose, where humans naturally choose predictable words for clarity. It's also weak on translated text, technical writing, and any genre where vocabulary is constrained. Modern detectors weight perplexity less than they did in 2023, treating it as one input among many. The full feature set we use is documented on /methodology.

How Perplexity Interacts with Related Metrics

Perplexity functions as the exponential of cross-entropy, meaning a model with cross-entropy of 3.0 yields a perplexity of approximately 20.09. This mathematical relationship positions perplexity as an interpretable transformation of the fundamental loss metric used during language model training. While cross-entropy measures the average number of bits needed to encode each token, perplexity converts this into an effective vocabulary size, answering the question of how many equally likely choices the model faces at each prediction step.

Burstiness and perplexity measure orthogonal properties of text. A passage can exhibit low perplexity (high predictability) while maintaining high burstiness (varied sentence complexity), or vice versa. Research from 2024 demonstrates that human academic writing often combines moderate perplexity scores with elevated burstiness, whereas AI-generated content tends toward low perplexity and low burstiness simultaneously. Detection systems that rely on perplexity alone achieve accuracy rates between 65 and 75 percent, but combining perplexity with burstiness and n-gram entropy raises accuracy above 85 percent in controlled studies.

Edge Cases and Known Limits

Perplexity measurements become unreliable when applied to domain-specific or technical writing. A graduate thesis in theoretical physics may produce lower perplexity scores than a chatbot's response about the same topic, not because the thesis is AI-generated, but because specialized terminology follows rigid conventions that maximize predictability. Models trained primarily on general web text assign unexpectedly high probability to formulaic academic phrasing, creating false negatives. Conversely, creative fiction with intentional stylistic experimentation can register high perplexity despite being human-authored.

Short text samples (fewer than 100 tokens) yield perplexity measurements with high variance and low diagnostic value. A single unusual word choice in a brief paragraph can spike the score disproportionately, while a 50-word AI-generated summary may fall within typical human ranges by chance. Multilingual texts present additional challenges, as code-switching between languages dramatically increases perplexity when evaluated by monolingual models. Practitioners working with non-English content or mixed-language documents report that perplexity thresholds calibrated on English corpora require substantial adjustment, often by factors of 1.5 to 2.0, to maintain comparable detection performance.