Skip to content
14 min readUpdated 29 October 2026

AI Watermark Detection: Methods, Access and Limits

Understand keyed text watermark detection, its access limits, and how it differs from generic AI-writing scores and hidden-character inspection.

If you paste text into an AI checker and get a percentage back, it is tempting to read the number as a fact about who wrote it. It isn't, and in most cases that percentage isn't watermark detection at all: generic checkers classify writing style. AI watermark detection is narrower. It looks for a statistical pattern that a particular generator may have embedded while choosing tokens, and even a positive result doesn't identify an author the way a fingerprint identifies a person.

That matters because an untouched passage, a shortened draft, a translation and a normally edited document can all produce different detector results. The useful question is not simply whether a watermark exists. It is how much evidence remains, what threshold the detector uses, and how much false-positive risk the operator accepts.

Table of Contents

What AI watermark detection tries to do

People who get a detector score usually want a simple answer: human or AI. The number can't give that certainty. It represents evidence under a particular model, calibration method and operating point chosen by the detector's provider or user.

A watermark detector searches for a token pattern created by the generator. It is not mainly looking for repetition, hedging, polished grammar or a recognizable "AI voice." Those stylistic features show up in human writing too and can disappear during editing. A statistical watermark detector instead checks whether the chosen tokens fit a hidden pattern that the generating system was built to favor, and it needs the provider's secret key to do so.

The signal is not a forensic stamp

The watermark is spread across the text. A detector collects weak clues from many token positions and asks whether the combined pattern looks more like watermarked generation than unwatermarked writing. The result is probabilistic from the start.

Google's SynthID documentation describes detection as probabilistic. Its detector can return watermarked, not watermarked or uncertain, rather than forcing every passage into a decisive verdict.

Treat a detector result as evidence for a decision, not as proof of authorship.

Benchmark results on untouched text show whether a method can recognize its own signal under controlled conditions. They don't tell you how much survives shortening, grammar fixes, tone changes, translation or a rewrite by a human editor. This article explains how the signal is planted, how a detector weighs it, and why ordinary editing can push the evidence below a chosen threshold.

How probabilistic watermarks like SynthID are built

A simplified way to picture it: a language model is about to choose the next token. A watermarking system uses a secret key and the preceding tokens to give some candidate tokens a small advantage over others. Each nudge is small enough to keep the text useful, but repeated across many choices it forms a pattern that a detector holding the same key can test for.

The key is not a visible mark. It decides which tokens get the preference at each position. Because that depends on earlier tokens and a private key, a reader sees ordinary text, not a signature.

Token choices create the pattern

SynthID-Text uses a scheme called Tournament sampling. Pseudorandom functions keyed on the preceding tokens score candidate tokens, and the sampling process favors higher-scoring ones. The detector later recomputes those scores for the text in front of it and checks whether they are higher on average than unwatermarked text would normally produce.

Other approaches, including Gumbel-max and inverse-transform sampling, use different maths but share the central idea: they adjust token selection so a hidden statistical preference builds up over a passage.

A process diagram explaining how probabilistic AI watermarks like SynthID function through generation and detection stages.

This signal is not attached after generation. The model creates it while choosing tokens, which is also its weakness: a later rewrite replaces some of the token choices that carried the evidence.

For a more technical walkthrough, see our guide to how AI text watermarks work. It separates token-choice watermarks from hidden Unicode characters, which are different problems with different checks.

A watermark needs enough scored text to separate signal from chance. A short passage may hold too little evidence for a stable decision, while a longer untouched passage gives the detector more positions to evaluate. More text doesn't make the result certain, but it gives the statistical test more to work with.

How a detector decides: watermarked, not watermarked or uncertain

The detector starts with the candidate text and the secret key used to build the watermark. At each position it recomputes the relevant scores, rates the observed choice, and combines the evidence across the passage. It then compares the total with what unwatermarked text would typically produce.

Google describes a Bayesian detector that turns that comparison into a decision using two threshold values, tuned to target particular false-positive and false-negative rates. The result is watermarked, not watermarked or uncertain, depending on where the evidence falls.

Why the grey zone is intentional

A passage near the boundary has weak or conflicting evidence. The uncertain state exists because forcing that passage into a binary category would hide how little the detector knows.

Detector output Statistical condition Practical meaning
Watermarked Evidence crosses the positive threshold The pattern is consistent enough with the watermark under the chosen calibration
Not watermarked Evidence falls below the negative threshold The detector doesn't find enough support for the watermark
Uncertain Evidence lies between the thresholds The passage doesn't justify a confident classification

This is why the true-positive rate at a fixed false-positive rate matters. A permissive threshold catches more watermarked passages but also flags more unwatermarked writing. A stricter threshold reduces false alarms but leaves more watermarked text unresolved.

The threshold is not a fixed property of SynthID. It is a policy choice by whoever operates the detector. The same score can produce different labels if the operator changes the tolerated error rates, text-length assumptions or decision boundaries.

Benchmark scores and why TPR at 1% FPR matters

A later theoretical analysis summarizes the original SynthID-Text evaluation at a fixed operating point. In the setup it describes, with 1,500 watermarked and 10,000 unwatermarked texts of 400 tokens each, generated with Gemma-7B, SynthID-Text reached a true-positive rate of 85% at a false-positive rate of 1%, compared with 73% for the prior state of the art. A later theoretical analysis of SynthID-Text summarizes that result from the Nature paper on SynthID-Text before running its own, differently configured experiments.

The comparison is meaningful because it holds the false-positive rate constant. Raw accuracy without an operating point can hide a policy problem: a detector that calls almost everything watermarked looks effective until you ask how often it wrongly flags unwatermarked writing.

What the benchmark does and doesn't establish

Detector TPR at 1% FPR
SynthID-Text in the reported Gemma-7B setup 85%
Prior state of the art in the same setup 73%

These two figures come from one setup with one model and one passage length. We don't have comparable figures for AUC, other false-positive rates, or perplexity-based and retrieval-based detectors, so they are not shown.

TPR at 1% FPR is a starting point for comparison, not a production guarantee. Real threshold selection depends on text length and the false-positive rate you can accept. Our methodology page explains how Simple Unmark treats these limits, including why we can't measure how much of a provider's watermark remains after a clean: we don't have access to providers' detectors.

Where detection breaks down in practice

How can a detector score perfectly on untouched text and struggle after an ordinary edit? One robustness study reported perfect baseline detection (F1 = 1.0) on unmodified text. Synonym substitution caused the mildest drop, reducing F1 to 0.884 at a high replacement ratio, while round-trip translation through Chinese reduced it to 0.711. The authors name paraphrasing, copy-paste modification and back-translation as meaning-preserving attacks the scheme is vulnerable to.

The reason is mechanical. A detector evaluates token choices, not meaning. Replace enough tokens and you replace the observations that supported the watermark. The paragraph may keep its idea while presenting a different statistical sequence, so the score moves toward the unwatermarked baseline.

Three practical failure modes

  • Short passages. Few scored tokens give limited evidence against ordinary variation, so a confident decision is harder to justify.
  • Mixed drafts. AI-generated sentences may sit beside human-written ones, so only part of the document carries the pattern. The combined score weakens when the watermarked portion is short.
  • Domain or language shifts. Calibration from one kind of text may not transfer to code-switched writing, specialist vocabulary or another text distribution.

A stronger watermark applies a bigger preference to each token, but the trade-off remains. A stronger preference may affect generation quality, and rewriting can still replace enough choices to reduce the total evidence. Strength is not the same as resilience.

A benchmark on untouched output describes the easiest case a detector will see. Real text usually has edits, mixed authorship and formatting changes.

The gap appears without any deliberate evasion. An editor who shortens a paragraph, changes its tone, translates it or fixes its grammar can alter the token distribution without meaning to. Evaluations should include these post-generation changes, and in practice a result is better treated as evidence at a chosen threshold than as a verdict about authorship.

What detectors can and cannot catch after editing

Editing has no single predictable effect. A typo fix may leave nearly every scored token in place, while a full rewrite can replace the sequence that carried the watermark. The detector sees only the final text, not its history, so the watermark alone can't tell a legitimate cleanup from an evasion attempt.

Changes that may preserve more evidence

Punctuation fixes, typo corrections, light reordering and short insertions of human text can leave much of the original token sequence intact. The detector may keep useful power in those cases, but the result still depends on passage length, the model and the threshold.

Changes that tend to erode evidence include synonym substitution, sentence-level paraphrasing, translation round trips and heavy shortening. In the robustness study above, even meaning-preserving synonym substitution lowered detection performance somewhat, and translation lowered it more.

There is no universal break point. It depends on the watermark design, the source model, the kind of text and how much of the passage was rewritten. Claims that a detector always survives "light editing," or always fails after a fixed amount of rewriting, go beyond the available evidence.

Why editing isn't symmetric

Removing surviving evidence is easier than rebuilding a watermark after it has been disrupted. An editor can replace tokens and lower the score, but the detector cannot infer which original tokens were removed or restore their contribution.

Sampling temperature also matters, because it changes how probability is spread across token choices, and detector behavior may change with it. The safe conclusion is modest: post-edit detection has to be tested on the actual workflow, not inferred from results on untouched output.

Hidden characters are a separate issue. Invisible Unicode left by copying can be removed as formatting, and you can check for it with the free, browser-only invisible character remover, but removing it does not touch a token-choice watermark. See hidden Unicode versus statistical watermarks for the difference.

What this means for editors, writers and compliance teams

An editor shouldn't treat a "watermarked" label as a final finding. Use it to decide what to look at next: check the revision history where you have one, verify facts, and consider whether human changes could affect the detector's calibration.

Detector access is also limited. As of 29 October 2026, Google has published SynthID-Text and a configuration-specific reference implementation, which is not a general checker for arbitrary Gemini text. Anthropic's detection API for its Claude watermark is in restricted private preview, and OpenAI limits textGrain detection to approved researchers and expert organizations. Most people who see a "watermark" score are therefore looking at something other than the provider's keyed detector.

Writers using AI assistance face a different decision. If disclosure matters, decide how you'll describe the assistance before heavy cleanup, translation or paraphrasing changes the text. Once the final version differs substantially from the generated one, a detector may no longer answer the original provenance question reliably.

A usable audit trail

Compliance teams need more than a coloured badge from an API. Record the original text, the final text tested, the detector and its version, the score, the text length, and the threshold or false-positive target used.

Then record the context that could change the interpretation:

  • Editing history: shortening, grammar correction, translation, paraphrasing and mixed authorship.
  • Risk level: more scrutiny for high-consequence decisions than for routine editorial triage.
  • Corroboration: provenance metadata, disclosure statements, revision records and factual review.
  • Vendor questions: which text lengths were validated, how uncertainty is reported, how thresholds were calibrated, and whether post-edit testing is available.

Keep the score and operating point, not only the final label.

Policy templates often reduce results to "AI detected" or "AI not detected," which throws away the information decision-makers need. A vendor should explain whether its output is a calibrated probability, a raw score, a thresholded state or something else. Whoever reviews the result should keep that evidence trail and make the uncertainty visible, so a statistical flag doesn't become an automatic penalty.

Ask about the threshold, not just whether the watermark is gone

"Is the watermark gone?" sounds like one technical question but hides several. Which detector was used? What length of text did it evaluate? What edits happened between generation and testing? Which false-positive rate did the operator accept?

A detector produces evidence on a continuous scale. The label comes later, when someone applies thresholds. As Google's documentation explains, the system can return an uncertain state because the evidence doesn't always support either strong conclusion.

The operating-point view

Question Binary framing Operating-point framing
What does the output mean? A verdict about whether AI wrote the text Evidence evaluated under a defined detector and threshold
What happens after editing? The watermark is either present or gone Editing changes the available signal and may move the score into uncertainty
How should risk be handled? The same label rule everywhere Thresholds chosen according to false-positive and false-negative costs
What should reviewers record? A yes-or-no result Score, text length, detector version, threshold and document context

Benchmark scores show what a detector can do under controlled conditions. Edits move the score, short text weakens the evidence, and mixed or shifted text strains calibration. A single pass or fail label hides all of that.

Treat watermark detection as one input among several. Provenance metadata, writing history, disclosure statements and fact-checking may all matter, depending on the decision. None of them turns a probabilistic signal into proof.

  • ai watermark detection
  • synthid
  • detector limits
  • paraphrasing attacks
  • ai text provenance

Related articles

13 min read

AI Detection Bypass Tools: Claims and Limits

Understand what bypass tools change, why classifier scores differ from watermark tests, and how to review rewritten text without false confidence.

Read post