Production AI Agent Architecture: From Tool Calls to Reliable Workflows

Agent versus deterministic workflow

A deterministic workflow has control flow written by the developer. An agent introduces model-driven decisions such as which tool to call, whether another step is needed, or when the task is complete. The additional flexibility is useful when the path cannot be fully enumerated in advance, but it also introduces variability and new failure modes.

Start with a deterministic workflow when the process is known. Add model-driven decisions only where they create measurable value. This makes testing, cost control, and debugging substantially easier.

• Ask whether the task requires dynamic control flow.
• Define a measurable success condition before adding autonomy.
• Prefer the simplest architecture that meets the requirement.

The orchestrator is the control plane

The orchestrator should own step limits, timeouts, retries, cost budgets, state transitions, and termination. These rules belong in code rather than in a prompt. A prompt can tell the model what to prefer; it cannot safely enforce a maximum spend or prevent a duplicate payment.

The orchestrator also records a trace for each model call and tool call. That trace becomes the raw material for debugging and evaluation.

• Maximum steps and wall-clock time.
• Maximum input/output tokens and cost per run.
• Explicit termination conditions and failure states.

Design tools as narrow contracts

Tool schemas are part of the agent interface. Use typed parameters, enums, required fields, sensible ranges, and clear descriptions. Tell the model when a tool should not be used. Avoid a catalog of overlapping tools because ambiguity increases selection errors.

Tool responses should be structured and actionable. A terminal business error should tell the model what happened so it can choose another path rather than returning a raw stack trace.

Memory without context pollution

Working memory belongs to the current run. Episodic memory captures previous events. Semantic memory stores durable facts. Procedural memory describes how the system operates through prompts, tools, and policies. Do not dump entire transcripts into long-term memory. Extract durable facts through a schema and retain source, timestamp, and confidence.

Long conversations also require compaction. Summarize old steps into a smaller state while keeping the original trace retrievable. The live prompt should contain what the next decision requires, not everything that has ever happened.

Retries, side effects, and human approval

Transient errors such as timeouts and rate limits can usually be retried with exponential backoff and a cap. Terminal errors such as invalid input or permission denial should not be retried blindly. Any side-effecting tool should be designed for idempotency so a network retry cannot execute the same action twice.

Human approval is most useful when an action is irreversible or has a large blast radius. Examples include money movement, deletion, external communication, or publishing customer-visible content. Present the reviewer with a clear proposed action or diff rather than an opaque reasoning trace.

Evaluate outcomes and trajectories

Agent evaluation needs more than final-answer similarity because the same goal can be reached through different valid paths. Measure task success, tool selection accuracy, argument validity, unnecessary steps, cost, latency, and safety violations. Store traces and replay representative tasks after prompt, model, or tool changes.

Silent success is particularly important: an agent may skip a required verification step and still produce a plausible answer. Step-level traces plus outcome checks make these failures visible.