Methods and evaluation
This page explains useful signal families, evaluation metrics, and public reference datasets. It does not publish a universal accuracy claim because performance changes by model, sample, language, transformation, and threshold.
These are common signal categories for interpreting detector output and designing representative evaluations. No individual signal proves authorship.
Text evaluation can combine statistical writing signals, classifier output, and robustness checks across editing and paraphrasing.
Code evaluation should separate common boilerplate from generator-associated patterns and test how results change after human edits.
Audio evaluation can examine spectral, timing, prosody, and channel signals across clean, compressed, and re-recorded samples.
Music evaluation can compare spectral repetition, timbre, structure, and metadata while accounting for mastering and distribution changes.
Video evaluation should test frame-level, temporal, face, and audio-video consistency across resolution and recompression changes.
| Dataset | Domain | Notes | Metric(s) |
|---|---|---|---|
| HC3 / HC3+ | Text | High-quality ChatGPT vs Human | AUROC, FPR@TPR |
| RAID | Text | Adversarial attacks & domains | Robustness Score |
| AIGCodeSet | Code | Python generation tasks | AUROC |
| ASVspoof 2019/21 | Audio | Logical/Physical Access | EER, min t-DCF |
| ADD 2022 | Audio | Audio Deepfake Detection | EER |
| DFDC | Video | 100k+ clips, Facebook backed | Video-AUC |
| FaceForensics++ | Video | Diverse manipulation methods | Frame-AUC |
| Celeb-DF | Video | High-quality Deepfakes | AUC |
These datasets are public references for designing evaluations. Listing a dataset does not claim a published ZeroTrue score on it.
AUROC
Across thresholds
Measures class separation across thresholds. Report false positives and calibration alongside it.
AUROC
By language
Results should be separated by programming language, source, and degree of human editing.
EER
Error tradeoff
Equal Error Rate summarizes the tradeoff between false acceptance and false rejection.
Clip AUC
Frame and clip
Report clip-level and frame-level behavior separately, including recompressed samples.
Illustrative labels for reading confidence with policy context.
Test held-out generator families, languages, media sources, and content domains.
Measure behavior after compression, cropping, paraphrasing, editing, replay, and re-recording.
Publish false positives, false negatives, confidence calibration, model version, and test date.