The promise is straightforward. Feed an AI model enough good experimental data and it can predict which catalysts will turn captured carbon dioxide into fuel, saving years of trial and error. A study led by SLAC National Accelerator Laboratory, published in Nature Catalysis on 31 July 2026, shows how that promise breaks in practice, and what it takes to repair it.
Four laboratories agreed to run the same catalyst test under the same protocol. They got four different results.
Key takeaways:
- Four US labs (SLAC, Penn State, Stanford and UC Santa Barbara) ran round-robin tests on the same rhodium-based catalyst.
- The goal was to generate reproducible training data for an AI model predicting catalyst performance.
- Initial results varied significantly in the amounts of carbon monoxide and methane produced.
- One of the largest sources of variation was how vigorously the mixture was stirred or shaken.
- After standardising reactor design, protocols and conditions, results converged.
- The paper offers guidance for anyone feeding real-world experimental data into machine-learning models.
Why catalysts, and why AI
Turning carbon dioxide into fuel requires catalysts that work fast, efficiently and for a long time. Most lab studies can only observe a catalyst for days, but real-world deactivation happens over months or years as impurities build up and temperatures cycle.
A well-trained AI model could bridge that gap. Enter temperature, reaction time and catalyst formulation; get a prediction of long-term performance; confirm with a handful of well-chosen experiments. The catch is that the model needs a large volume of high-quality, consistent data to learn from.
The round-robin experiment
To build that dataset, the four labs agreed on protocols and tested the same catalyst for a key first step: producing carbon monoxide from carbon dioxide.
When they compared notes, the numbers did not match. Each team reported different yields of carbon monoxide and different amounts of methane, an unwanted by-product. As the SLAC team put it, a computer cannot learn from four datasets that disagree about the same experiment.
"It was a bit of an eye-opener."
— Adam Hoffman, SLAC staff scientist and senior author, 3 August 2026
Finding the culprit
The teams went back through their methods step by step. They identified a handful of mismatches. One of the biggest turned out to be mundane: how hard the reaction mixture was stirred or shaken.
With further standardisation across the four sites, the results began to converge. The paper sets out recommendations for improving reproducibility, including consistency in:
- Reactor design
- Operating protocols
- Experimental conditions
The lesson for anyone building AI models
This study is about catalysis, but the message is universal and it is the oldest rule in data science: a model is only as good as the data you feed it.
| Where the problem shows up | What it looks like |
|---|---|
| Multi-site data collection | Same protocol, different equipment, subtly different outcomes |
| Merged datasets | Hidden inconsistencies that the model learns as if they were signal |
| Long-horizon prediction | Small early variations compound into large errors |
In an era of "AI scientist" headlines, this paper is a reminder that the unglamorous work of standardising how data are generated may matter more than the choice of model.
Concept to take away: data provenance. Before you trust a dataset, ask how it was produced, by whom, with what equipment, and under what conditions. If the answer differs across sources, your model will inherit the disagreement. This applies to lab data, to web-scraped text, and to business records alike.
Funding and facilities
The work was supported in part by the US Department of Energy Office of Science. The SLAC portion took place at the Stanford Synchrotron Radiation Lightsource. Testing equipment was supplied in part by Co-ACCESS, part of the SUNCAT Center for Interface Science and Catalysis.
