Building AI Agents: A Practical Guide to Production-Ready Systems

Key takeaways

• Build around one bounded workflow with a clear owner and measurable outcome.
• Give the agent narrow, typed tools, durable state, a bounded loop, and approval gates for consequential actions.
• Test the complete run, then move repeatable work into deterministic apps so the agent reasons once and the software runs the step consistently.

What building an AI agent actually means

OpenAI's definition is the cleanest one in circulation: "Agents are systems that independently accomplish tasks on your behalf." The same guide is blunt about what does not qualify. "Applications that integrate LLMs but don't use them to control workflow execution ... are not agents," which rules out chatbots, single-turn calls, and classifiers. (A Practical Guide to Building Agents)

Anthropic draws the line in the same place with sharper vocabulary. Workflows are "systems where LLMs and tools are orchestrated through predefined code paths." Agents are "systems where LLMs dynamically direct their own processes and tool usage." (Building Effective Agents)

So the artifact you are building is a loop. A model receives a goal, chooses an action, calls a tool, reads the result, updates state, and decides whether to continue or stop. Everything else in this article is a decision about what surrounds that loop.

The first decision is the task boundary, and it carries more weight than the model choice. An agent that triages support tickets or reconciles a payment against a contract can be tested against a fixed set of cases. An agent that is supposed to "run finance ops" cannot be tested at all, because nobody can enumerate what it is meant to do. Draw the boundary at a workflow with an owner, an input, and an outcome someone would notice if it were wrong.

Start with the workflow, then pick the model

Pick a process before picking a stack. Write down the normal path, the exceptions, the steps that cannot be undone, and the person who answers for the result. That document tells you whether you need retrieval, code execution, database writes, or a human approval gate.

Anthropic's guidance here is worth taking literally: "we recommend finding the simplest solution possible, and only increasing complexity when needed," and this "might mean not building agentic systems at all." A workflow with a known order of steps does not need an agent. If your process is deterministic on paper, keep it deterministic in code and use the model for the one step that requires interpretation.

Model selection should run against representative tasks from that workflow rather than a leaderboard. Test quality, latency, structured output reliability, tool-call accuracy, and cost on your own cases. Then keep the model layer replaceable. Route simple classification to a small model and reserve stronger reasoning for ambiguous cases. A build that cannot survive a model swap has a short shelf life.

Tools, state, and the loop

Tools are the boundary between reasoning and consequence, and they deserve more design attention than the prompt. Anthropic reported that on SWE-bench, "we actually spent more time optimizing our tools than the overall prompt."

Give each tool a narrow schema, explicit permissions, validation, and errors the model can act on. A function like create_refund(order_id, amount) is safer and easier to evaluate than a general database tool that can write anywhere. OWASP names the failure mode directly. LLM06:2025 Excessive Agency is "the vulnerability that allows damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction," with a root cause of "excessive functionality; excessive permissions; excessive autonomy." Their mitigation is to "avoid the use of open-ended extensions where possible (e.g., run a shell command, fetch a URL, etc.) and use extensions with more granular functionality." (OWASP LLM06:2025)

Tool count matters less than tool overlap. OpenAI observed that "some implementations successfully manage more than 15 well-defined, distinct tools while others struggle with fewer than 10 overlapping tools."

State is where most prototypes quietly break. A context window is a transcript, and a transcript is not a record. Durable state belongs in a database or object store the application owns: workflow status, prior attempts, approvals, the evidence behind a decision. Anthropic's research-system writeup puts the stakes plainly. "Agents are stateful and errors compound," and because "restarts are expensive and frustrating for users," they built systems that "can resume from where the agent was when the errors occurred." (How we built our multi-agent research system)

Instructions carry the rest of the contract. State the purpose, the data the agent can see, the rules on each tool, the output format, and the conditions under which it stops. An exit condition can be a closed ticket, a validated record, a human approval, or a cap on iterations. Without one, a tool that keeps returning the same error turns into an open-ended bill.

The loop itself needs timeouts, bounded retries, idempotency keys, and a defined failure path. Retries are the specific place where side effects duplicate. Before retrying a payment, a message, or a record update, check whether the first attempt landed. Every run needs an identifier that ties model decisions, tool calls, results, and errors together, because without one you cannot answer a question about what happened last Tuesday.

Guardrails, approvals, and the audit trail

Guardrails work in layers. OpenAI's tool safeguard guidance is the most useful piece of it: "Assess the risk of each tool available to your agent by assigning a rating ... based on factors like read-only vs. write access, reversibility, required account permissions, and financial impact." Those ratings drive what pauses for a check and what escalates to a person.

Two triggers should route to a human. OpenAI names them as "exceeding failure thresholds" and "high-risk actions," the latter covering work that is "sensitive, irreversible, or have high stakes," with examples including "canceling user orders, authorizing large refunds, or making payments." Approval should record who approved what, when, and against which evidence.

Enforcement belongs below the model. OWASP's complete-mediation guidance says to "implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not." An instruction in a prompt is a preference. A permission check in code is a control.

The audit question is the one that gets agent projects stopped after deployment. OWASP's agentic threat taxonomy gives it a name, T8 Repudiation and Untraceability, which "occurs when actions performed by AI agents cannot be traced back or accounted for due to insufficient logging or transparency in decision-making processes." Their mitigation asks for "complete logging, cryptographic verification, enriched metadata, and real-time monitoring." (Agentic AI: Threats and Mitigations, v1.1, December 2025)

NIST's AI Risk Management Framework, which is voluntary rather than binding, offers the sharpest way to test your own logs: "Transparency can answer the question of 'what happened' in the system. Explainability can answer the question of 'how' a decision was made in the system. Interpretability can answer the question of 'why' a decision was made by the system." (NIST AI RMF 1.0) Most agent traces answer the first question and stop. Record the instruction version, the tool inventory available at decision time, the arguments, the result, and the approval, or you will be reconstructing March from Slack messages.

What most agent builds get wrong

The expensive mistake is reaching for multiple agents before proving one cannot do the job. Anthropic measured the cost: "agents typically use about 4× more tokens than chat interactions" and "multi-agent systems use about 15× more tokens than chats." Their conclusion is that multi-agent designs "require tasks where the value of the task is high enough to pay for the increased performance." OpenAI reaches the same place from the design side: "maximize a single agent's capabilities first." A coordination diagram is not evidence.

The second mistake is treating chat history as memory. It works until the conversation ends, the window fills, or two runs need to see the same record.

The third is evaluating the answer rather than the run. Build a test set that includes edge cases, policy violations, tool failures, and ambiguous requests, then measure task success, tool selection, safe refusal, approval compliance, latency, and cost. A model can write an excellent response while the surrounding system writes to the wrong account.

This article does not cover evaluation methodology in any depth, and it should not. Scoring agent behavior on open-ended tasks is a harder problem than anything above, and it deserves separate treatment.

How do you build an AI agent?

1. Pick one bounded workflow with a named owner and a measurable outcome.
2. Write the normal path, the exceptions, and the irreversible steps before choosing a model.
3. Select a model by testing representative tasks from that workflow, and keep the model layer replaceable.
4. Expose the minimum set of narrow, typed tools, each with its own permissions and validation.
5. Implement a bounded loop with timeouts, capped retries, idempotency keys, and a defined failure path.
6. Put durable state in an application database rather than the context window.
7. Rate each tool by risk and gate the high-risk ones behind human approval that records evidence.
8. Instrument every run with a trace that captures decisions, tool calls, arguments, results, and approvals.
9. Build an evaluation set covering failures and policy violations, and run it before every change.
10. Read the traces, find the steps the agent reasons through identically every time, and compile those into code.

Step ten is the one that separates a working prototype from a system that stays affordable at volume.

What we're building at Major in response

Our position is that the repeatable stretch of an agent workflow should stop being a model problem. When the inputs, rules, and outputs of a step are stable, re-deriving that step on every run buys nothing and costs tokens, latency, and variance. Major is the enterprise platform where agents build the software they run on: when an agent works out how to handle a repeatable part of a task, it builds an app for that part, with a managed database, storage, scoped permissions, and audit logs at the platform layer. From then on it runs the app instead of reasoning through the step again. Reason once. Run forever.

That single move settles three of the problems this article has been circling. The work becomes deterministic because it is code. It becomes stateful because the app owns the records instead of a context window that disappears. It becomes governable because permissions and logs attach where the action happens, which is what OWASP T8 and NIST's "what happened" test are both asking for.

Be clear about what this does not fix. The model still reasons, and the judgment calls remain probabilistic, so a compiled app makes the repeatable half predictable and leaves the interpretive half to be evaluated like any other model output. It does not solve agent evaluation. It does not make a badly scoped workflow succeed. Draw the boundary badly and you will get a deterministic app that does the wrong thing reliably.

If your traces show the same reasoning repeating run after run, that is the step worth compiling. You can see how Major turns a repeated agent step into a governed app with its own database, permissions, and logs.

Related articles

• AI workflow builder: build workflows that keep running
• Multi-agent orchestration: patterns and what they cost
• LLM observability: what to instrument when the work is a black box

Related articles

• AI Workflow Builder: Build Workflows That Keep Running
• LLM Cost Management: Control Spend by Running Less AI
• LLM Routing: How It Works and What the Benchmarks Show