(Author here) Posted this because a thread was going around with secondhand rejection stories and no named source, and I wanted to know if any of it actually held up. It does, more than I expected — ICLR's own retrospective admits a "significant false positive rate" in their automated checker, ACL desk-rejected 100+ papers and then reversed the policy mid-cycle after a meeting with their executive, and an independent 48k-paper audit puts real hallucinated-citation rates at 18.7%–34.9% of accepted papers depending on venue — with reviewer scores showing essentially zero correlation with catching it.
I also pulled one specific fabricated citation another party (GPTZero) had published and re-checked it against Crossref myself rather than taking their word for it. It isn't there either.
Disclosure: I run strictcite.com, a citation checker mentioned briefly at the end of the piece — flagging that upfront rather than having it come up in the comments.
(Author here) Posted this because a thread was going around with secondhand rejection stories and no named source, and I wanted to know if any of it actually held up. It does, more than I expected — ICLR's own retrospective admits a "significant false positive rate" in their automated checker, ACL desk-rejected 100+ papers and then reversed the policy mid-cycle after a meeting with their executive, and an independent 48k-paper audit puts real hallucinated-citation rates at 18.7%–34.9% of accepted papers depending on venue — with reviewer scores showing essentially zero correlation with catching it.
I also pulled one specific fabricated citation another party (GPTZero) had published and re-checked it against Crossref myself rather than taking their word for it. It isn't there either.
Disclosure: I run strictcite.com, a citation checker mentioned briefly at the end of the piece — flagging that upfront rather than having it come up in the comments.