Study Finds Chemical Chain-of-Thought in Reasoning Models Prone to Hallucination

A new arXiv paper examines how language models trained for chemical reasoning use chain-of-thought steps, and finds that the intermediate reasoning frequently contains fabricated content. Testing four reasoning model families across twelve chemistry tasks, the authors report that hallucination is widespread and largely disconnected from the final answer. The work suggests chain-of-thought traces in this domain act more like an unreliable scratchpad than a faithful record of the model's reasoning.