Authors
Jieli Zhou, Naiwu Chen, Longzhang Liu, Peiyu Zhang* (AI Chemistry Group, Future Chemistry Department, XtalPi Inc).
Abstract
AI models, including large language models, are increasingly integrated into scientific discovery workflows, yet they remain prone to hallucination. In experimental sciences, such errors translate directly into failed wet-lab validations and wasted resources; in self-improving agentic systems, confident errors risk being reinforced rather than corrected. Retrosynthesis provides a representative example of this failure mode: existing models can generate chemically plausible routes, but cannot reliably determine which routes are experimentally feasible.
We define Chemical Hallucination as a route that appears valid yet fails under competing reactive sites, unresolved selectivity, or missing mechanistic support, a failure largely invisible to the Recall@K metric. We introduce SureRoute, a chemical verifier-anchored retrosynthesis platform that suppresses Chemical Hallucination. SureRoute combines a multi-model ensemble, data asset retrieval, and ChemHarness, an executable chemical intuition engine for route verification and reliability-first ranking.
On a benchmark of 350 real-world industrial targets, SureRoute reaches 51.7% recall@1, 1.7–2.5× that of seven single-step models and three frontier LLMs, while cutting top-1 Chemical Hallucination to 4.6%, a 4–6× reduction relative to frontier LLMs. As a model-agnostic reranker, ChemHarness drives detectable hallucination toward near-zero across arbitrary backbone candidates. SureRoute shows that reliable scientific AI requires not only strong generation, but executable verification.
Recall@1/@3 across 7 SOTA single-step retrosynthesis models, 3 frontier LLMs, and SureRoute (Lite: ensemble; Max: full system). SureRoute’s recall@1 is a step-function jump above every other method.
SureRoute converts retrosynthesis from confidence-ranked generation into verifier-anchored self-improvement. Existing retrosynthesis systems typically generate candidate routes and rank them by model confidence, which can surface a plausible but chemically hallucination route. SureRoute instead uses a four-layer architecture: a generation layer builds a diverse candidate pool; ChemHarness verifies each candidate for competition, selectivity, mechanism, and precedent; the ranking layer prioritizes chemically verified routes over confident hallucinations; and the evolution layer converts expert-confirmed failures into high-quality data.
Full PDF:
