Hidden phrases buried inside 24,371 research papers just caught more than 500 peer reviewers cheating with AI.
The International Conference on Machine Learning (ICML) 2026, one of the field's top venues, ran an unprecedented sting operation on its own reviewers. Program chairs embedded invisible watermarks into submitted papers: two phrases selected from a dictionary of 170,000 options, placed where only an LLM processing the document would see them. The probability of guessing the correct pair by chance? Less than one in ten billion.
When a reviewer fed a paper into ChatGPT, Claude, or any other model and submitted the output as their review, those telltale phrases showed up in the text. The trap worked with over 80% accuracy in pre-submission testing against frontier models, and every flagged case was manually verified by a human before action was taken.
The Numbers
Out of roughly 24,371 submissions (more than double last year's 12,107), ICML desk-rejected 497 papers and flagged 795 reviews from 506 unique reviewers. The harshest penalty hit 51 reviewers who had more than half their reviews flagged. All their reviews were deleted and they were permanently removed from the reviewer pool.
Here's the twist: ICML offered reviewers a choice. Policy A banned all LLM use, period. Policy B allowed LLMs for understanding papers and polishing prose. Reviewers who chose the strict no-LLM policy, then used LLMs anyway, got caught. They agreed to the rules, then broke them.
The rejected papers weren't necessarily bad science. They were desk-rejected because their authors were also serving as reviewers who violated the policy. ICML's rules allow "cascading desk rejections" where peer-review abuse triggers rejection of all papers by any author of an offending paper.
A Bigger Problem Than One Conference
ICML isn't an outlier. A 2025 Frontiers survey found that more than half of researchers admitted to using AI in peer review. At ICLR 2026, a separate analysis estimated 21% of peer reviews were AI-generated. GPTZero found over 50 confirmed hallucinated citations (completely fabricated references) across 300 scanned ICLR papers, with some of those papers scoring 8 out of 10 from human reviewers who missed the fake citations entirely.
The peer review system is supposed to be the quality filter for published science. When reviewers outsource their judgment to the same models that helped write the papers, that filter breaks down. ICML's watermark approach is clever, but it's also an arms race. Models will eventually learn to ignore hidden instructions, and watermark dictionaries will need constant updating.
For anyone building products on top of published AI research, this is a credibility problem. If a paper's citations might be fabricated and its reviews might be machine-generated, how much weight should you give its claims? The 497 rejected papers are the ones that got caught. The real number is almost certainly higher.