papersSEP 10 04:00 UTC
Study finds LLMs degrade as error auditors with batch size, hallucinating confidently
Researchers assembled a corpus of 150 academic papers with deliberately planted errors to test how well large language models can act as automated document-quality auditors. They report that detection reliability worsens as processing batch sizes increase, and that models sometimes fabricate audit findings with high confidence. The results cast doubt on deploying LLMs unsupervised for contamination-detection tasks.