Why Two AI Text Detectors Disagree
Two text detectors can assign different labels to the same passage because their scores, thresholds, definitions, and input conditions may differ. Here is how to compare the results without treating the larger number as a winner.
In this article
Two text detectors can give different labels for the same passage without either displayed number settling the disagreement. The practical next step is to keep the exact passage, then compare what each tool says its score and label mean. Do not rank the results by the larger percentage.
A score is a numerical output. A label is a decision made using a tool's own rule, which may include a threshold. If two tools use different scales, definitions, or thresholds, their scores are not a shared reliability ranking.
A score and a label are different things
A tool might display a number and then apply a cutoff: above the cutoff receives an AI-related label; below it does not. Another tool can use a different cutoff, even when its displayed number looks similar.
The following values are hypothetical teaching values. They are not results from any detector, and they are not probabilities.
| Tool | Score | Threshold for an AI label | Result under that tool's rule |
|---|---|---|---|
| Tool A | 0.65 | 0.60 | AI label |
| Tool B | 0.68 | 0.75 | No AI label |
Tool B displays the larger score, 0.68 rather than 0.65. But 0.68 does not reach Tool B's threshold of 0.75, so Tool B does not apply its AI label. Tool A's lower score passes Tool A's lower threshold, so it does apply the label.
This example does not show that Tool A is stricter, that Tool B is more accurate, or that either score is more trustworthy. It shows only the arithmetic of two different decision rules. A score from one tool is not automatically comparable with a score from another merely because both are displayed as decimals or percentages.
Why calibration changes how a number should be read
Calibration concerns how closely a reported probability matches the frequency of the outcome across comparable examples with known origins. If a tool displays 0.70, it can be tempting to read that as a 70% probability that AI wrote the passage. That interpretation may not be justified.
NIST AI 700-1, printed page 31, discusses poorly calibrated discriminator scores and the risk of treating them as probabilities, including when a fixed classification threshold is used. The point for your comparison is simple: a number can be useful within a tool's result, while still not functioning as a portable probability or a common confidence scale across tools.
This NIST discussion evaluates participating systems. It is not a ZeroTrue benchmark, does not establish ZeroTrue score calibration, and does not tell you an error rate for your passage.
The text version matters as much as the displayed result
Detector results can also differ because the inputs are not truly the same. A shortened excerpt removes context and changes the amount of text under review. A translation changes wording and structure. Editing can change sentences, transitions, vocabulary, and the balance of material in the passage.
These are not minor housekeeping details. Comparing a full English draft with a shortened, translated, or heavily edited version is a comparison of different inputs. It cannot answer whether two tools disagreed about the same passage.
The BUST benchmark paper reports substantial variation in detector performance across writing tasks in its evaluated dataset. That finding does not evaluate ZeroTrue or establish how any detector will perform on your text. It does provide a reason not to generalize casually from one kind of writing to another. A result concerning a polished essay, for example, is not automatically a result concerning a translated message, a short product description, or an edited excerpt.
Make a comparison record before drawing a conclusion
Use a short record that separates what you know from what the tools do not disclose. Fill in only details you can see or verify.
| Item to compare | Tool A | Tool B |
|---|---|---|
| Exact input | Full passage copied exactly | Full passage copied exactly |
| Displayed score and label | Record both exactly as shown | Record both exactly as shown |
| Score or label definition | Copy the available definition | Copy the available definition |
| Version and conditions | Record only what is displayed | Record only what is displayed |
| Missing details | Note unavailable threshold, version, or definition | Note unavailable threshold, version, or definition |
The first row determines whether you are comparing like with like. If the inputs match and labels conflict, the useful conclusion is that the tools produced conflicting outputs for that exact passage. That is a reason to inspect the definitions and the source context, not to declare the larger score the winner.
If the inputs differ, label the comparison accordingly: for example, “full original versus shortened translation.” You may still learn something about how the outputs changed, but you do not have a direct same-passage disagreement.
If definitions, thresholds, or versions are unavailable, leave those fields unavailable. Do not infer a hidden threshold from a label, invent a model version, or convert a percentage into a probability. Your record is still useful because it states precisely what cannot be compared.
What to do with conflicting labels
When the same input produces conflicting labels, return to the question you need to answer. If you need to know whether a claim in the text is true, check the source supporting that claim. If you need to understand how the text was produced, look for relevant source material, drafts, or disclosure that you can actually verify. A detector disagreement does not turn either question into a numerical contest.
If you still need a first check of a particular passage, ZeroTrue's text detector can provide a result for that input. Record the submitted text and displayed result together. If you already have conflicting results, adding ZeroTrue does not by itself settle the disagreement. For help reading a first result, see how to read an AI detector score.
Does no AI label prove that a person wrote the text?
No. It tells you only that the tool did not apply that label to the submitted input under its available rule. It does not establish authorship, identity, or the process used to create the text.
Will more scans resolve the disagreement?
Not by themselves. Repeatedly submitting altered excerpts can create more outputs without producing a fair comparison. A further scan is useful when you keep the exact passage fixed and record what the result means. It is not useful as score shopping for a preferred answer.