Most AI agents you see do small jobs: sort the email, summarize the meeting, draft the reply. That isn’t because the AI agent isn’t smart enough for more. It’s because an AI agent built from a handful of rules has no structure around it, so the AI agent has to carry everything in its head. That works while the job is shallow. As the job gets bigger, you need what any big job has always needed: agreed records, agreed processes, and code for the parts that should never vary. The size of the job an AI agent can handle is set by the structure around it.
The same questions apply to a sales pipeline, a purchase order or a month-end close: what stays fixed, what needs judgment, and who decides?
What the AI field has found
The widely read industry guide “Building Effective Agents” draws the main line. In a workflow, the steps are written in code in advance and each one is initially analyzed by AI. In an agent, the model decides its own steps. Their advice is to use the simplest design that works.
A widely used industry guide to building agents says where AI earns its place: work that needs nuanced judgment, rule sets that have grown too tangled to maintain, and messy, unstructured information. For everything else, “a deterministic solution may suffice.” In other words: code where the work is fixed, AI where it isn’t.
And one AI lab, writing about long jobs, recommends that agents keep structured notes in files outside the conversation and read them back, because the conversation alone won’t hold them.
From the shop floor: one machine versus a plant
A skilled operator can run one machine from memory. They know its quirks, the settings that work, what to watch for. Nobody runs a whole plant that way.
At one plant, every part we ran had its own setup sheet: six to eight hundred of them, pulled into one system people could actually use. The presses ran on PLC programs that did exactly the same thing every cycle. Procedures said who did what, in what order. And on top of all that, people’s judgment handled what the system couldn’t: the odd part, the new problem, the call nobody had written down.
That’s the division that works:
- Records in an agreed format, so anyone can find and trust them.
- Processes everyone follows, so the order doesn’t depend on memory.
- Code for what never varies, because it’s fast, cheap and never gets tired.
- People’s judgment for everything else.
AI slots into that last line. It doesn’t replace the rest.
Six ways to build an AI agent
The real question behind any AI agent design is who decides the next step: the code, or the AI agent. Here are the main designs, from the most structure to the least.
| Design | Who decides the next step | Good for | Weak spot |
|---|---|---|---|
| Fixed workflow | The code; AI does the initial analysis at each step | Repeatable jobs: sort, extract, draft | Breaks when the job doesn’t fit the script |
| Router | The AI agent sorts the request; the code sends it to the right handler | Front desks, mixed inboxes | Only as good as the sorting |
| State machine with AI inside | The code holds the states and allowed moves; the AI agent works within each state | Long jobs with gates, where skipping a step is costly | More building up front |
| Procedures, skills and principles | The AI agent, guided by written procedures, with its state in files | Long, varied work, like a project over months | The gates are words, not locks |
| Autonomous agent | The AI agent, given a goal and tools | Open-ended work like coding and research | Drift, cost, hard to check |
| Several agents | A lead AI agent hands pieces to helper AI agents | Big research jobs that split into parallel pieces | Cost: one AI lab measured about fifteen times the usage of a chat |
An AI agent that’s just a long list of rules is really the fifth design with flavor on top. For a small, forgiving job like sorting email, that’s fine. For the same job, the first or second design is usually better: cheaper, and it does the same thing every time.
The third design deserves more attention than it gets. It’s the PLC state machine, applied to AI: the code knows which state the job is in and only allows the defined moves; the AI agent does the thinking inside each state; and to move on, it has to pass a check the code enforces.
Two kinds of knowledge
As jobs get bigger, the structure that matters most is knowledge, and there are two kinds:
- Reference knowledge: manuals, standards, past jobs. The AI agent looks things up. (People often call this retrieval, or RAG.)
- State: the project’s own records, where things stand, what’s decided, what’s open, in agreed formats. Simple files work: a spreadsheet, a CSV or a JSON file, as long as everyone agrees on the fields.
An AI agent can work without a reference library. It can’t run a months-long job without reliable state.
The A.N. Creation Office keeps its state this way, with more than thirty record templates. Here is the header row of its action log:
ID, Opened, Action, Who, Why / from, Due, Status, Done how, Closed, Notes
Every action has the same fields, and the status is one of a fixed list. Any session, any AI agent and any person can pick up the log and know exactly where things stand.
The line keeps moving
Two cautions, so this doesn’t overreach.
Some AI agents are small because small is right. Building structure costs time. A two-line AI agent that writes alt text for an image is the correct design. Missing structure explains why big jobs fail on simple AI agents; it doesn’t mean every small AI agent is missing something.
The ceiling keeps rising. Each generation of AI can hold more in its head, so a rules-only AI agent can take on a little more each year. But the gap doesn’t close, for the same reason a great operator still doesn’t run a plant from memory. Bigger, longer, costlier jobs need records, processes and code around the judgment.
Try this
Take one job you’d like an AI agent to do and split it into three lists. What never varies (calculations, checks, file handling): that’s code. What the job needs to remember (status, decisions, numbers): that’s records, with agreed fields. What takes judgment (messy input, unusual cases, the conversation): that’s the AI agent. If the third list is the only one you’ve written down, that’s why the AI agent drifts.
Sources
- “Building Effective Agents” (Dec 2024): source
- “A practical guide to building agents” (2025), page 6: source
- “Effective context engineering for AI agents” (29 Sep 2025): source
- “How we built our multi-agent research system” (June 2025): source
- A.N. Creation Office, action log template: ancreationoffice.com/template-action-log.csv Free to use under the MIT License.
These pages sit with the rest of the Advance Now family. The hotbotspot tool library writes the methods up for AI. A.N. Creation Office is where the public AI agents run as open experiments.