Benchmark report · Detector v10-ft-g · November 2026
How accurate is ParaTrace?
Every figure on this page was measured on text the detector never saw in training, scored by the production service at one fixed setting, the Strict operating point.
99.8 %
of AI-written answers detected
25,601 answers · 90 models · 26 developers
1.0 %
of human texts wrongly flagged
22,681 held-out texts, same setting
95.9 %
still detected under evasion attacks
11 attack types · 16,500 texts
161 ms
to score a 300-word text
on our servers
Detection
It catches 99.8 % of AI-written answers
25,601 answers to real user prompts, written by 90 models from 26 developers, at a setting that wrongly flags about 1 in 100 human texts.
| Developer | Detected | Answers |
|---|---|---|
| OpenAI | 99.6 % | 4,509 |
| 99.8 % | 3,696 | |
| Anthropic | 99.8 % | 3,380 |
| Meta | > 99.9 % | 2,120 |
| Mistral | 99.8 % | 1,933 |
| Alibaba | 99.9 % | 1,767 |
| DeepSeek | 99.9 % | 1,371 |
| Reka | 99.9 % | 1,190 |
| Microsoft | 99.8 % | 1,062 |
| 01.AI | 99.9 % | 896 |
| Cohere | 99.9 % | 813 |
| xAI | 99.8 % | 561 |
| Zhipu | 100 % | 371 |
| Amazon | 100 % | 307 |
| NVIDIA | 100 % | 304 |
| Nexusflow | 100 % | 299 |
| MiniMax (small sample) | 100 % | 179 |
| Databricks (small sample) | 100 % | 166 |
| Moonshot (small sample) | 99.2 % | 131 |
| Tencent (small sample) | 100 % | 54 |
| Other developers | 100 % | 492 |
† Fewer than 200 answers, so indicative only. “Other developers” groups six smaller developers, a research fine-tune and one anonymous test model. Snapshots and reasoning modes of one model count once.
Late-2025 flagship models
- Anthropic99.6 % of 284
- Google100 % of 633
- xAI99.0 % of 102
- OpenAI98.8 % of 336
Each developer’s leading model as of late 2025. Answers were collected up to October 2025. Newer models are not yet benchmarked.
New models it never saw
99.4 %
of 1,116 answers from late-2025 model versions that were not in its training data.
Developers it never saw
99.9 %
of 3,917 answers from developers with no model at all in its training data.
Clear-cut verdicts
97 %
of AI-written answers score 99 % or higher, far from the cut-off, so most verdicts don’t hinge on a close call.
False alarms
About 1 in 100 human texts wrongly flagged
1.04 % of 22,681 held-out human texts were flagged as AI. From 200 words up it is 0.6 %, and most false alarms get the milder label, AI-assisted.
- Under 50 words in the text, 4.1 % (35 of 846 texts)
- 50–99 words in the text, 1.6 % (30 of 1,909 texts)
- 100–199 words in the text, 1.3 % (87 of 6,821 texts)
- 200–399 words in the text, 0.6 % (54 of 9,017 texts)
- 400+ words in the text, 0.7 % (30 of 4,088 texts)
Under 50 words, 4.1 % are wrongly flagged, which is why the service warns on short texts. From 200 words it is 0.6 %.
| Writing | Flagged | Texts |
|---|---|---|
| Legal texts (small sample) | 0.0 % | 148 |
| Patents (small sample) | 0.0 % | 115 |
| Medical notes (small sample) | 0.0 % | 17 |
| Résumés (small sample) | 0.0 % | 15 |
| Essays | 0.1 % | 5,562 |
| Reviews | 0.3 % | 1,602 |
| Blogs (small sample) | 0.5 % | 189 |
| Scientific papers | 0.7 % | 432 |
| Scientific abstracts | 0.9 % | 1,816 |
| Questions & answers | 0.9 % | 3,369 |
| Emails (small sample) | 1.3 % | 75 |
| News articles | 1.5 % | 2,733 |
| Forum posts | 1.5 % | 1,859 |
| Government reports (small sample) | 1.6 % | 61 |
| Encyclopedia articles | 1.7 % | 1,718 |
| Educational texts (small sample) | 1.7 % | 116 |
| Other web writing | 1.9 % | 2,037 |
| Speeches and debates (small sample) | 3.7 % | 81 |
| Book excerpts | 3.8 % | 443 |
| Recipes | 5.8 % | 293 |
† Small sample, so indicative only.
These are held-out documents, never used in training, from the same kinds of sources the detector learned from. Writing unlike those sources can be flagged more often. See Where to be careful below.
Evasion
Common evasion tricks cost it little
Across 11 common attacks, 95.9 % of AI texts are still detected, against 97.7 % before any attack. For text from chat assistants it is 98.5 %.
- AI paraphrasing, 91.1 % of 1,500 texts, 95 % confidence interval 89.5–92.4 %
- Deleted articles (a, an, the), 94.1 % of 1,500 texts, 95 % confidence interval 92.8–95.2 %
- Deliberate misspellings, 95.1 % of 1,500 texts, 95 % confidence interval 93.9–96.1 %
- Random upper/lower case, 95.3 % of 1,500 texts, 95 % confidence interval 94.1–96.2 %
- Synonym swaps, 96.4 % of 1,500 texts, 95 % confidence interval 95.3–97.2 %
- British/American spellings, 96.5 % of 1,500 texts, 95 % confidence interval 95.5–97.3 %
- Look-alike letters, 97.1 % of 1,500 texts, 95 % confidence interval 96.2–97.9 %
- Invisible characters, 97.1 % of 1,500 texts, 95 % confidence interval 96.2–97.9 %
- Extra spaces, 97.1 % of 1,500 texts, 95 % confidence interval 96.2–97.9 %
- Extra paragraph breaks, 97.1 % of 1,500 texts, 95 % confidence interval 96.2–97.9 %
- Changed numbers, 97.5 % of 1,500 texts, 95 % confidence interval 96.6–98.2 %
1,500 texts per attack. Lines show 95 % confidence intervals, and the axis starts at 88 %.
The hardest attack
91.1 %
of AI texts rewritten by an AI paraphrasing model are still caught.
Character tricks
Look-alike letters, invisible characters and extra spaces are undone before scoring, so they cost it less than one point.
Independent documents
The attacked texts come from a public robustness benchmark. None of these documents were used in training.
Beyond training
It holds up beyond its training data
Detection that only works on familiar material is brittle. These tests use genres, models and settings the detector was never shown.
A genre it never studied
90.4 %
of 5,000 AI-written poems detected, although poetry was absent from training. For poems from chat assistants it is 97.5 %.
Models held back on purpose
99.7 %
of 8,118 texts from two widely used model versions that were deliberately kept out of training.
Any sampling setting
94.2 %
for text generated with random sampling, the hardest setting. With a repetition penalty it is 99.3 % to 99.9 %.
Formats
Short or long, prose or code
From 50 words up, detection stays high whatever the length or format of the answer.
| Length | Detected | Answers |
|---|---|---|
| 50–99 words | 99.2 % | 2,982 |
| 100–199 words | 99.7 % | 5,071 |
| 200–399 words | 99.9 % | 8,491 |
| 400–799 words | > 99.9 % | 6,690 |
| 800 words or more | 99.9 % | 2,367 |
| Content | Detected | Answers |
|---|---|---|
| Prose | 99.8 % | 20,571 |
| With code | 100 % | 2,197 |
| With math notation | 99.9 % | 1,552 |
| With tables | 100 % | 796 |
| With no formatting at all | 99.5 % | 6,106 |
It isn’t just spotting formatting. Answers with no headings, lists or bold text at all are detected 99.5 % of the time.
Assisted or generated
AI-assisted or AI-generated
Every flagged text is also called AI-assisted (human writing that AI edited or paraphrased) or AI-generated.
| Flagged texts | AI-generated | AI-assisted | Texts |
|---|---|---|---|
| AI-written answers | 99.0 % | 1.0 % | 25,553 |
| AI texts under evasion attacks | 97.4 % | 2.6 % | 15,818 |
| AI-paraphrased human writing | 70.6 % | 29.4 % | 618 |
| Human texts wrongly flagged | 24.2 % | 75.8 % | 236 |
Shares of the texts flagged in each test, so every row adds up to 100 %.
AI answers called AI-generated
99.0 %
of the 25,553 flagged AI-written answers were called AI-generated, the stronger label.
False alarms get the milder label
75.8 %
of the 236 human texts wrongly flagged were called AI-assisted, not AI-generated.
Speed
Fast enough to check everything
A text is scored in a fraction of a second, even a long one.
161 ms
to score a 300-word text
756 ms
to score a typical 1,500-word text
Time on our servers from text in to verdict out, text preparation included. Each figure is the median of 60 runs, one text at a time. Network and queue time are not included.
Limits
Where to be careful
What we measured that falls short, and what we haven’t measured yet.
- Persuasive essays by adults, a kind of writing kept out of training entirely, were wrongly flagged 20.1 % of the time (35 of 174). Human writing unlike the training sources can be flagged far more often than the averages above.
- Human poetry was wrongly flagged 1.4 % of the time, across 1,733 poems.
- Short texts under 50 words were wrongly flagged 4.1 % of the time. The service warns on anything shorter.
- Synonym-swapping tools can mislead it. Human text run through one was flagged 37.6 % of the time.
- Humanizer tools, sold to make AI text pass as human, are its weakest spot. See how it compares with Pangram on them.
- AI-paraphrased human writing counts as AI. By design, 52.2 % of it was flagged, because the final wording came from a model. 29.4 % of those texts were called AI-assisted.
- English only. We have not yet evaluated documents that mix human and AI writing, human text lightly edited with AI, or paraphrasing tools beyond those tested here.
- Newer models are not yet benchmarked. Answers were collected up to October 2025.
This is a statistical estimate, not proof. Don't use it as the sole basis for decisions about a person.
Method
How we measured
- One setting throughout.
- Every figure uses the Strict operating point, calibrated so that about 1 in 100 human validation texts is flagged. Longer texts are judged part by part, as when you submit them.
- AI-written answers.
- 25,601 responses to real user prompts from public chat logs, collected between 2024 and October 2025, plus a small set generated in-house. Answers to a prompt that is also in the training data were left out.
- Human writing.
- 22,681 held-out documents, never used in training, from 43 sources that include essays, questions and answers, news, forum posts, reviews, encyclopedia articles, scientific papers and abstracts, legal texts, patents and government reports.
- Evasion and new genres.
- Documents from a public robustness benchmark, none used in training. Each attack was applied to the same 1,500 AI texts.
- AI-assisted or AI-generated.
- The service calls every flagged text one or the other, by how large its estimated AI share is. The shares here are of the texts it flagged in each test.
- Scoring.
- Every text was scored by the production service running detector v10-ft-g, text preparation included, exactly as when you submit it.
- Against other detectors.
- We set ParaTrace beside Pangram on the public tests Pangram reports on, in a separate comparison, and repeated the check NeurIPS ran on scientific papers.
- Rates.
- The detection rate is the share of AI-written texts flagged as AI. The false-alarm rate is the share of human-written texts flagged as AI. Ranges are 95 % confidence intervals (Wilson).