ARTICLE · FROM THE FIELD · Natural experiment

The experiment you already ran

People ask me what AI is going to do for their plant, and underneath, whether it’s going to cost half their people their jobs. My answer usually starts in a filing cabinet. Most plants are sitting on years of test results, quality logs and process data that nobody has ever really analyzed. That’s the low-hanging fruit. You don’t need cutting-edge AI to pick it. You need the data you already have, and someone who can tell whether the answer is real.

The same questions apply to a sales pipeline, a purchase order or a month-end close: what stays fixed, what needs judgment, and who decides?

What the AI field has found

A widely used industry guide to building agents names three kinds of tools an AI agent needs. The first is data: tools to “retrieve context and information,” like querying databases or reading documents. Everything else depends on the AI agent having the right information in front of it. And its list of where AI adds the most value includes work with “heavy reliance on unstructured data,” the messy records that rule-based systems can’t handle.

There’s a matching caution. AI would rather give an answer than say “I don’t know,” and it can sound sure when it’s wrong. So the same habit that applies to any analysis applies here: check the answer against something you know is true before you change anything.

From the shop floor: the natural experiment

You come into a plant thinking you need a designed experiment: hold everything still, change one thing at a time, measure. Then you find that someone has been collecting data for ten years.

Over those ten years the process changed for a hundred reasons: new resin lots, setups drifting, weather, maintenance, new equipment. Nobody ran a formal experiment. The plant was the experiment. I call it a natural DOE. Look back far enough, and you start to see the process window.

A step change in the data, and you go ask: “Oh yeah, we put a new chiller in that month.” Cycle times creeping up, and it’s the water lines plating over and running hot. A plant-wide jump of about ten psi on the water system, and a surprisingly big effect on production.

Then there’s the folklore. Maintenance ran the cooling water at 45 psi. Where did that number come from? A plumber told somebody, a long time ago. The data said a higher pressure would have given more flow through the molds, more even cooling and quicker cycles, as long as the pipe sizes could carry it. You have to look at the data. You have to do the analysis.

What AI changes

The classic way to analyze a pile of test data is real work: export it, clean it, set it up in a statistics package, run the studies, check the significance. Weeks, for someone who knows which tests to run.

Now, if the data is saved and readable, you can put it in front of an AI agent and ask, “What is this telling me?” It will try different cuts, look for patterns and suggest what to check, and weeks of exploration happen in minutes. Many plants don’t even know they should be doing this analysis. Now it’s cheap enough that there’s no excuse.

What AI doesn’t replace is judgment. Someone still has to know whether a pattern is real on the plant floor or an artifact of how the data was logged. That’s the person who remembers the chiller.

How it carries over

  • Start with the data you have. Test results, scrap logs, downtime records, setup sheets, quote history. Polluted? Often. Useless? Rarely.
  • Ask what it’s telling you, then ask why. Treat every finding as a question to check, not a conclusion.
  • Look for step changes and seasons. Many plant mysteries hide in the time dimension: they happen in summer, or after a certain week.
  • Check against the floor. Before anyone changes a process, someone who knows it confirms the finding is real.
  • Write down what you learn. The A.N. Creation Office’s troubleshooting principle says it plainly: “Write each change and its result down, so the next try starts from evidence.”

As for the jobs question: most of this is work that was never getting done at all. AI isn’t replacing the person who answers the front door. It’s replacing the blank stare at a cabinet of test logs.

Try this

Find one set of data your plant collects and nobody analyzes. Make a copy, take out anything sensitive, and ask an AI agent: “What is this data telling me? What changed over time, and when?” Take the two most interesting answers to the person who knows that process best, and ask them whether it’s real.

Sources

These pages sit with the rest of the Advance Now family. The hotbotspot tool library writes the methods up for AI. A.N. Creation Office is where the public AI agents run as open experiments.