Scientists Can Now Build Their Own Tools. Here’s the Hidden Risk.

• 11 June, 2026
• By Dave Cassel

Scientists have always had a complicated relationship with code. Some wrote their own analysis scripts from scratch, spending hours on work that had nothing to do with the actual science. Others depended on developers who didn't fully understand the problem, and hoped the resulting tool was close enough.

AI coding assistants have changed both of those situations. Scientists can now build better tools faster, whether they were already coding or not.

Stanford's Center for Open and REproducible Science (CORES) held its annual symposium this spring, bringing together more than 200 researchers to wrestle with what AI means for scientific reproducibility. The write-up is worth a read.

CORES Faculty Director Russell Poldrack put the opportunity plainly: "Automated software coding is a major development that enables scientists to run more complex experiments and analyze data faster, at scale, and with greater accuracy."

He also named the risk in the same breath: faster code generation is not the same as better code.

Most scientists have no formal training in software engineering. Until recently, that was a manageable limitation. They wrote their own scripts. Those scripts might be slow and sometimes brittle, but at least the scientist understood exactly what the code was doing.

Now, AI coding tools can produce in minutes what used to take days. But as Poldrack noted: "This can also go wrong in unexpected ways.... How do you validate automatically generated code, to make sure it's giving you the right answers?"

That is not a rhetorical question. It's an open problem.

There's a deeper issue underneath it. Poldrack described it this way: "AI systems are not trained with [scientific] values. Instead, they're trained to give people answers that they like."

An AI coding assistant will produce code that runs, looks reasonable, and returns a result. It won't tell you when the result is wrong. A scientist without a software engineering background may not catch it either.

For commercial R&D companies, this is a business risk, not just an academic one.

In a university lab, a reproducibility failure is an embarrassment and a setback. In a commercial R&D company, it can mean shaky intellectual property, failed audits, or results that don't hold up when a partner or investor looks closely.

The scientists at your company are not doing anything wrong by using AI coding tools. They should be using them. The question is whether you have the systems in place to catch the errors those tools introduce.

That means version control for analysis code, lightweight code review processes (even informal ones), and a clear record of what code produced what result.

Software engineers wouldn't ship production code without testing and review. Scientific analysis code is making decisions about your data. It warrants the same discipline.

That's the gap a Fractional CTO fills.

Most lab directors are scientists first. Bringing software engineering discipline into a research environment isn't their job, and it's not something most R&D organizations think to prioritize until something goes wrong.

The goal is not to put gates in front of scientists. The goal is to make sure the tools they build are trustworthy enough to stake your IP on.

Share this post: