There is a pattern I keep encountering in industrial IT conversations. Someone describes a process problem, and the proposed solution is an agent. Not because an agent is the right tool, but because it is the current one.
I am not against agents. I build with them. But the technology conversation has run ahead of the design conversation, and the result is a lot of unnecessary complexity being deployed into operations that needed something much simpler.
The useful skill right now is not knowing how to build an agent. It is knowing when not to.
Four levels, in ascending order of cost
Almost every process problem people bring me can be placed on this ladder. The discipline is to start at the bottom and justify every step upward.
Level 1 — A rule. Deterministic logic, no AI. If the supplier is on the approved list and the amount is under the threshold, approve it. Rules are free to run, trivially auditable, and never hallucinate. A surprising share of “AI opportunities” are actually a business rule that nobody wrote down.
Level 2 — A workflow. Orchestrated steps across systems, still deterministic. This is where something like n8n earns its place: fetch from the API, transform, validate, write to the ERP, notify. No model involved. Reliable, observable, debuggable.
Level 3 — AI inside a workflow. The workflow stays deterministic; a model handles one bounded, unstructured step within it. Extract the fields from this document. Classify this ticket. Summarize this thread. The model is a component, not the architect, and the surrounding workflow still governs what happens with its output.
Level 4 — An agent. The model decides which steps to take, in what order, using which tools. Genuinely powerful for open-ended tasks. Genuinely expensive to run, hard to audit, and non-deterministic in ways that matter when something goes wrong at 2am.
Most business process problems in an ERP-driven operation live at Levels 1 through 3. I have seen very few where Level 4 was the correct choice rather than the interesting one.
The diagnosis that prevents wasted build
Before choosing a level, answer one question honestly: is the problem unstructured input, or is it unclear procedure?
These look similar from a distance and have opposite solutions.
If the input is unstructured — a messy email, a scanned form, a free-text message — but you know exactly what to do with it once it is structured, you have a Level 3 problem. Use a model for extraction, wrap it in a workflow, and you are done. This is the most common and most successful pattern I deploy.
If the input is structured but the procedure is unclear — the process varies by customer, by person, by phase of the moon, and nobody can articulate the rule — then you do not have an AI problem. You have a process definition problem. A model will not fix this. It will automate the inconsistency and make it harder to debug. Go map the process first, and you will usually find a Level 1 or Level 2 solution waiting.
Teams that skip this diagnosis build agents to paper over undefined processes, and then cannot explain why the agent behaves unpredictably. It behaves unpredictably because the process is unpredictable. The model is faithfully reflecting the organization.
What agents are genuinely good at
I want to be fair to Level 4, because there are real cases where it is correct.
Exploratory work with no fixed path. Research tasks, initial investigations, “find out everything we know about this customer” — work where the steps genuinely cannot be enumerated in advance and the cost of a wrong turn is low.
Developer and analyst tooling. Coding assistants, query builders, data exploration. The user is technical, can evaluate the output immediately, and the blast radius of an error is small and reversible.
Triage and routing. Deciding which queue this belongs in, given a broad and evolving taxonomy. The failure mode is a misroute, which is cheap, rather than a wrong transaction, which is not.
Note the common thread: low consequence per action, and a human who can evaluate the result. Where those two conditions fail — high consequence, unable to verify — agents should not be making the decision.
Designing for the failure you will actually have
The mistake in most agent projects is not the model choice. It is the absence of a containment strategy.
An agent that can call tools and write to systems needs, at minimum:
- Bounded permissions. Give it the narrowest access that accomplishes the task. An agent that can write to the customer master does not need to be able to delete from it.
- A dry-run mode. Every capability should be exercisable without side effects, so you can see what it would have done before letting it do it.
- Full action logging. Not just the input and output, but the intermediate decisions and tool calls. When something goes wrong, and it will, this is the only way to reconstruct what happened.
- A kill switch that a non-engineer can operate. If stopping the system requires a developer at a keyboard, you have not finished building it.
- Cost and rate ceilings. Agent loops can burn tokens and API quota remarkably fast. Cap them by design, not by monitoring.
An agent running in production without these is not an experiment. It is an outage waiting for a trigger.
The evaluation problem nobody budgets for
Here is the question I ask teams who want to deploy an agent, and the one that most reliably reveals whether the project is ready: how will you know it got worse?
If there is no answer, the project is not ready, regardless of how well the demo went. Non-deterministic systems drift — models update, prompts accumulate tweaks, input distributions shift. Without an evaluation harness, you will not notice a regression until a user complains, and by then you will not know which change caused it.
For most operational use cases this does not require anything sophisticated. A set of representative examples with known-correct outputs, run on every change, is enough to catch the majority of regressions. Teams skip it because it is unglamorous and produces no visible feature. It is also the single highest-value thing you can build alongside the system itself.
A pragmatic sequence
If you are starting now, this sequence has served me well across ERP and integration work:
- Write down the process as it actually runs, including the exceptions. Most of the AI value is often already visible at this step, as rules.
- Build the deterministic workflow first, with the model-shaped step stubbed out as a manual queue. This proves the plumbing and gives you a working system while you work on the hard part.
- Replace the stub with a model call for that one bounded step. Measure before and after.
- Only if the problem is genuinely open-ended and low-consequence, consider letting the model plan the steps.
- Build the evaluation harness alongside step 3, not after step 4.
The appeal of this sequence is that every stage is independently useful and independently revertible. At no point do you have a half-built agent that does nothing.
The honest summary
Agents are a real capability, and they are overapplied. In industrial operations — where consequences are high, processes are constrained, and auditability is not optional — the reliable wins are at Levels 1 through 3: rules, workflows, and models in a bounded role inside a deterministic system.
Reach for Level 4 when the problem genuinely has no fixed path and a wrong turn is cheap. Otherwise, the simpler design will be easier to run, cheaper to operate, and possible to explain to an auditor.
The interesting engineering question is not “can we build an agent?” It is “what is the least sophisticated thing that solves this reliably?” Answer that honestly and you will ship more, break less, and spend considerably less time debugging something non-deterministic at two in the morning.
Anna Nurhayanto has spent 25+ years delivering enterprise systems and integration projects — ERP (Odoo) implementation, automotive dealer systems, and workflow automation with n8n — with current focus on practical AI/LLM application in industrial operations.