ARTICLE · FROM THE FIELD · Records · code · judgment

Structure sets the size of the job

Most AI agents you see do small jobs: sort the email, summarize the meeting, draft the reply. That isn’t because the AI agent isn’t smart enough for more. It’s because an AI agent built from a handful of rules has no structure around it, so the AI agent has to carry everything in its head. That works while the job is shallow. As the job gets bigger, you need what any big job has always needed: agreed records, agreed processes, and code for the parts that should never vary. The size of the job an AI agent can handle is set by the structure around it.

The same questions apply to a sales pipeline, a purchase order or a month-end close: what stays fixed, what needs judgment, and who decides?

What the AI field has found

The widely read industry guide “Building Effective Agents” draws the main line. In a workflow, the steps are written in code in advance and each one is initially analyzed by AI. In an agent, the model decides its own steps. Their advice is to use the simplest design that works.

A widely used industry guide to building agents says where AI earns its place: work that needs nuanced judgment, rule sets that have grown too tangled to maintain, and messy, unstructured information. For everything else, “a deterministic solution may suffice.” In other words: code where the work is fixed, AI where it isn’t.

And one AI lab, writing about long jobs, recommends that agents keep structured notes in files outside the conversation and read them back, because the conversation alone won’t hold them.

From the shop floor: one machine versus a plant

A skilled operator can run one machine from memory. They know its quirks, the settings that work, what to watch for. Nobody runs a whole plant that way.

At one plant, every part we ran had its own setup sheet: six to eight hundred of them, pulled into one system people could actually use. The presses ran on PLC programs that did exactly the same thing every cycle. Procedures said who did what, in what order. And on top of all that, people’s judgment handled what the system couldn’t: the odd part, the new problem, the call nobody had written down.

That’s the division that works:

  • Records in an agreed format, so anyone can find and trust them.
  • Processes everyone follows, so the order doesn’t depend on memory.
  • Code for what never varies, because it’s fast, cheap and never gets tired.
  • People’s judgment for everything else.

AI slots into that last line. It doesn’t replace the rest.

Six ways to build an AI agent

The real question behind any AI agent design is who decides the next step: the code, or the AI agent. Here are the main designs, from the most structure to the least.

DesignWho decides the next stepGood forWeak spot
Fixed workflowThe code; AI does the initial analysis at each stepRepeatable jobs: sort, extract, draftBreaks when the job doesn’t fit the script
RouterThe AI agent sorts the request; the code sends it to the right handlerFront desks, mixed inboxesOnly as good as the sorting
State machine with AI insideThe code holds the states and allowed moves; the AI agent works within each stateLong jobs with gates, where skipping a step is costlyMore building up front
Procedures, skills and principlesThe AI agent, guided by written procedures, with its state in filesLong, varied work, like a project over monthsThe gates are words, not locks
Autonomous agentThe AI agent, given a goal and toolsOpen-ended work like coding and researchDrift, cost, hard to check
Several agentsA lead AI agent hands pieces to helper AI agentsBig research jobs that split into parallel piecesCost: one AI lab measured about fifteen times the usage of a chat

An AI agent that’s just a long list of rules is really the fifth design with flavor on top. For a small, forgiving job like sorting email, that’s fine. For the same job, the first or second design is usually better: cheaper, and it does the same thing every time.

The third design deserves more attention than it gets. It’s the PLC state machine, applied to AI: the code knows which state the job is in and only allows the defined moves; the AI agent does the thinking inside each state; and to move on, it has to pass a check the code enforces.

Two kinds of knowledge

As jobs get bigger, the structure that matters most is knowledge, and there are two kinds:

  • Reference knowledge: manuals, standards, past jobs. The AI agent looks things up. (People often call this retrieval, or RAG.)
  • State: the project’s own records, where things stand, what’s decided, what’s open, in agreed formats. Simple files work: a spreadsheet, a CSV or a JSON file, as long as everyone agrees on the fields.

An AI agent can work without a reference library. It can’t run a months-long job without reliable state.

The A.N. Creation Office keeps its state this way, with more than thirty record templates. Here is the header row of its action log:

ID, Opened, Action, Who, Why / from, Due, Status, Done how, Closed, Notes

Every action has the same fields, and the status is one of a fixed list. Any session, any AI agent and any person can pick up the log and know exactly where things stand.

The line keeps moving

Two cautions, so this doesn’t overreach.

Some AI agents are small because small is right. Building structure costs time. A two-line AI agent that writes alt text for an image is the correct design. Missing structure explains why big jobs fail on simple AI agents; it doesn’t mean every small AI agent is missing something.

The ceiling keeps rising. Each generation of AI can hold more in its head, so a rules-only AI agent can take on a little more each year. But the gap doesn’t close, for the same reason a great operator still doesn’t run a plant from memory. Bigger, longer, costlier jobs need records, processes and code around the judgment.

Try this

Take one job you’d like an AI agent to do and split it into three lists. What never varies (calculations, checks, file handling): that’s code. What the job needs to remember (status, decisions, numbers): that’s records, with agreed fields. What takes judgment (messy input, unusual cases, the conversation): that’s the AI agent. If the third list is the only one you’ve written down, that’s why the AI agent drifts.

Sources

These pages sit with the rest of the Advance Now family. The hotbotspot tool library writes the methods up for AI. A.N. Creation Office is where the public AI agents run as open experiments.