Agents & Tools

Two engineers are asked to implement "find me two seats under R$300 for Saturday and hold them".

The first writes a workflow: extract date, quantity and max_price from the message with one model call that returns JSON; call search_events and search_seats in code; pick the cheapest adjacent pair; call hold_seats; ask the model to phrase the answer. Two model calls, every step testable, latency predictable.

The second writes an agent: give the model the tools and the goal, and let it decide what to call, in which order, and when to stop. It handles requests the first design never anticipated ("…and if there is nothing on Saturday, try Sunday, but not in the upper ring"), and it fails in ways the first design never could (it holds the wrong event, or loops forever comparing seats).

Both are correct designs — for different requests. This page is about the machinery of the second one, and about knowing when to choose the first.

Workflows and agents

The difference is who decides the next step: your code, or the model¹.

Start at the top of the table and move down only when the simpler level measurably fails on your evaluation set. Most production "agents" in 2026 are workflows with one or two agentic steps.

The agent loop

An agent is a loop around a model call. In the ReAct formulation² the model alternates reasoning (what do I know, what do I need), acting (requesting a tool call) and observing (reading the tool result), until it can answer.

flowchart TD
    start(["user message"]) --> ctx["build context:<br/>system prompt + tools + memory + history"]
    ctx --> call["call the model (via LLM gateway)"]
    call --> decide{"tool calls<br/>in the reply?"}
    decide -- no --> answer(["final answer to the user"])
    decide -- yes --> guard["Safety Mesh:<br/>validate + authorize each call"]
    guard --> run["execute tools (in parallel if independent)"]
    run --> append["append calls + results to history"]
    append --> budget{"steps, tokens, time<br/>within budget?"}
    budget -- yes --> call
    budget -- no --> stop(["stop: partial answer or hand off to a human"])

The loop itself is ordinary code. Here is a minimal, provider-agnostic version — the three collaborators are the other palette components:

import json

MAX_STEPS, MAX_TOKENS = 8, 40_000


def run_agent(llm, registry, guard, user, question, history):
    """llm: LLM-gateway client; registry: Tool Registry; guard: Safety Mesh.
    All three are this sketch's own interfaces, not a specific SDK."""
    messages = [*history, {"role": "user", "content": question}]
    used = 0
    for step in range(MAX_STEPS):
        reply = llm.chat(alias="assistant-default", system=SYSTEM_PROMPT,
                         messages=messages, tools=registry.schemas_for(user))
        used += reply.input_tokens + reply.output_tokens
        messages.append(reply.message)          # the assistant turn, with its tool calls
        if not reply.tool_calls:                # no action requested: this is the answer
            return reply.text
        for call in reply.tool_calls:           # the model may request several at once
            verdict = guard.check_tool_call(call.name, call.arguments, user)
            if verdict == "allow":
                result = registry.call(call.name, call.arguments, as_user=user)
            else:                               # denied, or needs the user's confirmation
                result = {"error": verdict}
            messages.append({"role": "tool", "tool_call_id": call.id,
                             "content": json.dumps(result)})
        if used > MAX_TOKENS:
            break
    return "I could not finish this request. A human agent will contact you."

Five details in those 25 lines carry most of the engineering:

1. The model never executes anything. It returns a request (tool_calls); the orchestrator validates, authorizes as the user (as_user=user) and executes.
2. Errors go back to the model as data. A denied or failed call becomes a tool result ({"error": ...}), so the model can recover — ask the fan, choose another seat — instead of crashing the loop.
3. Every loop has budgets: steps, tokens and (in production) wall-clock time. Without them, a confused model loops until the quota or the invoice stops it.
4. The tools offered depend on the user (schemas_for(user)): an anonymous visitor does not even see hold_seats. A tool the model cannot see is a tool it cannot misuse.
5. History grows at every step — the quadratic token growth computed in the overview.

# 2. the agent loop: at most max_steps model calls; tools in between
answer = ""
for step in range(settings.max_steps):
    last_step = step == settings.max_steps - 1
    stream = await router.acompletion(
        model=route, messages=messages, stream=True,
        tools=None if last_step else [t.spec() for t in tools.REGISTRY.values()],
        stream_options={"include_usage": True},
    )
    chunks = []
    async for chunk in stream:
        chunks.append(chunk)
        served_by = getattr(chunk, "model", None) or served_by
        delta = chunk.choices[0].delta if chunk.choices else None
        text = getattr(delta, "content", None) if delta else None
        if text:
            if first_token_at is None:
                first_token_at = time.monotonic()
            answer += text
            yield "token", text
    message = litellm.stream_chunk_builder(split_tool_calls(chunks), messages=messages)
    if message.usage:
        usage["prompt_tokens"] += message.usage.prompt_tokens or 0
        usage["completion_tokens"] += message.usage.completion_tokens or 0
    reply = message.choices[0].message
    calls = reply.tool_calls or []
    if not calls:
        break
    messages.append({"role": "assistant", "content": reply.content or "",
                     "tool_calls": [c.model_dump() for c in calls]})
    for c in calls:  # tools run between model calls; results go back as data
        result = await tools.call(c.function.name, c.function.arguments, ctx)
        tool_log.append(c.function.name)
        messages.append({"role": "tool", "tool_call_id": c.id, "name": c.function.name, "content": result})

Stopping and context management

As the history grows, context engineering keeps it useful: tools return compact results (5 seat options, not 5,000 seats); older turns are compacted into a summary; large documents are referenced by ID and fetched only when needed. A focused context is cheaper, faster, and more accurate than a long one³.

Tool calling

Tool calling (also called function calling) is the contract between the model and your code:

1. The request declares the tools: name, description and a JSON Schema of the arguments.
2. The model replies either with text or with one or more tool calls: {id, name, arguments}, where arguments should match the schema. Many APIs can enforce the schema with constrained decoding (structured outputs), so the JSON is always well-formed.
3. The orchestrator executes the call and sends the result back in a message linked to the call id.
4. The model continues with the result in its context.

The model chooses tools by reading their names and descriptions, so the tool interface is a prompt and an API at the same time. Good tools are designed for the model, not copied from the internal API:

Tools are side effects

A model call can be retried freely. A tool call changes the world. If hold_seats times out and the loop retries, did the first hold happen? This is the integration problem of Compute & services, and it has the same answers:

• Idempotency keys. The orchestrator derives a key from (conversation, step, call) and passes it to every write tool; the service returns the same result for the same key.
• Reversible first. Prefer actions with a natural undo — a hold expires in 10 minutes; an order with payment does not.
• Compensation. If a later step fails, run the inverse action (release_hold), exactly as in a saga.
• Risk tiers decide what may run without a human:

@dataclass
class Context:
    """Per-request facts the tools may use (identity comes from the gateway, never from the model)."""
    user_email: str | None
    evidence: set[str] = field(default_factory=set)  # ids of the documents the model has seen
async def get_order_status(args: GetOrderStatusArgs, ctx: Context) -> dict:
    if not ctx.user_email:
        return {"error": "the user is not logged in: ask them to log in to see their orders"}
    async with httpx.AsyncClient(base_url=settings.order_url, timeout=5) as http:
        r = await http.get(f"/orders/{args.order_id}")
    if r.status_code == 404:
        return {"error": "order not found"}
    r.raise_for_status()
    order = r.json()
    if order.get("customerEmail", "").lower() != ctx.user_email.lower():
        return {"error": "order not found"}  # same answer as 404: do not reveal that it exists
    return {"id": order["id"], "status": order["status"], "showId": order["showId"], "amount": order["amount"]}

Durable execution

A conversation that waits for the fan's confirmation, or a task that runs for minutes, cannot live only in a process's memory: pods restart. Persist the agent state after every step — messages, pending tool calls, idempotency keys — in a database, and resume from the last checkpoint. For long multi-step tasks, a durable workflow engine (Temporal-style: each step recorded, retried and resumable) gives you this for free; for short chats, a conversation_state row plus the session cache in the Memory Fabric is enough.

Planning

The agent loop plans one step at a time. For longer tasks, explicit planning helps:

Why every step must be checked

If each step is right with probability \(p\), independently, an \(n\)-step task succeeds with probability \(p^n\):

# Probability that an n-step task succeeds when each step succeeds with
# probability p, independently: P(task) = p ** n
steps = [1, 3, 5, 10, 20]
head = ["Per-step success"] + [f"{n} step{'s' if n > 1 else ''}" for n in steps]
print("<table><thead><tr>" + "".join(f"<th>{h}</th>" for h in head) + "</tr></thead><tbody>")
for p in (0.99, 0.97, 0.95, 0.90):
    cells = [f"{p:.0%}"] + [f"{p ** n:.0%}" for n in steps]
    print("<tr>" + "".join(f'<td style="text-align:right">{c}</td>' for c in cells) + "</tr>")
print("</tbody></table>")

A step that is right 95% of the time sounds good; a 10-step task built from it succeeds only 60% of the time. The levers are the same as for availability in series (Estimation): fewer steps (better tools, workflows for the known parts), validation after each step (schemas, business rules in code), and recovery (errors returned to the model, re-planning) — the agent equivalent of retries and redundancy.

Tool registry and MCP

Without a standard, every agent needs a custom integration with every tool: N agents × M tools. The Model Context Protocol (MCP)⁵ does for agents what the Language Server Protocol did for code editors: each tool provider writes one MCP server, each agent host embeds MCP clients, and any client can use any server — N + M integrations.

flowchart LR
    subgraph Host["Host: assistant service (orchestrator)"]
        c1["MCP client"]
        c2["MCP client"]
        c3["MCP client"]
    end
    c1 -- "Streamable HTTP + OAuth" --> s1["MCP server<br/>catalog team"]
    c2 -- "Streamable HTTP + OAuth" --> s2["MCP server<br/>inventory team"]
    c3 -- "stdio (local process)" --> s3["MCP server<br/>local files / dev tools"]
    s1 --> catalog["catalog service"]
    s2 --> inventory["inventory service"]

The essentials of the protocol:

• Roles. The host is the application (the BoxOffice assistant); it runs one client per connection; each server exposes capabilities.
• Server primitives. Tools (actions the model may call), resources (data the application can read into context, identified by URIs), and prompts (reusable templates offered to the user). Clients may in turn offer servers capabilities such as sampling (the server asks the host's model for a completion) and elicitation (the server asks the user for input).
• Messages. JSON-RPC 2.0. A session starts with initialize (version and capability negotiation); then tools/list discovers tools and tools/call invokes them; servers can notify tools/list_changed.
• Transports. stdio for a server running as a local subprocess; Streamable HTTP for remote servers, with OAuth 2.1-based authorization.

The inventory team publishes its capabilities as an MCP server. With the official Python SDK, a tool is a typed, documented function — the SDK derives the JSON Schema from the type hints and the description from the docstring:

# Official MCP Python SDK (package "mcp"); check its docs for the current API.
from mcp.server.fastmcp import FastMCP

from inventory_client import InventoryClient  # the team's existing HTTP client

mcp = FastMCP("boxoffice-inventory")
inventory = InventoryClient("http://inventory:8080")


@mcp.tool()
def search_seats(event_id: str, quantity: int, max_price_brl: float) -> list[dict]:
    """Find available seats for an event, adjacent when possible, priced at or
    below max_price_brl each. Returns at most 5 options, cheapest first."""
    return inventory.search(event_id, quantity, max_price_brl, limit=5)


@mcp.tool()
def hold_seats(event_id: str, seat_ids: list[str], idempotency_key: str) -> dict:
    """Hold seats for 10 minutes so the fan can pay. Reversible: an unpaid hold
    expires. Calling again with the same idempotency_key returns the same hold."""
    return inventory.hold(event_id, seat_ids, ttl_seconds=600, key=idempotency_key)


if __name__ == "__main__":
    mcp.run(transport="streamable-http")

On the wire, this is what the orchestrator's client sees when it discovers the tool (tools/list result, abridged) and then calls it:

{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "tools": [
      {
        "name": "hold_seats",
        "description": "Hold seats for 10 minutes so the fan can pay. Reversible: an unpaid hold expires. Calling again with the same idempotency_key returns the same hold.",
        "inputSchema": {
          "type": "object",
          "properties": {
            "event_id": {"type": "string"},
            "seat_ids": {"type": "array", "items": {"type": "string"}},
            "idempotency_key": {"type": "string"}
          },
          "required": ["event_id", "seat_ids", "idempotency_key"]
        },
        "annotations": {"readOnlyHint": false, "destructiveHint": false, "idempotentHint": true}
      }
    ]
  }
}

{"jsonrpc": "2.0", "id": 7, "method": "tools/call",
 "params": {"name": "hold_seats",
            "arguments": {"event_id": "evt_8812", "seat_ids": ["C12", "C13"],
                          "idempotency_key": "conv_51f3:step3:call1"}}}

{"jsonrpc": "2.0", "id": 7,
 "result": {"content": [{"type": "text",
                         "text": "{\"hold_id\": \"h_77a\", \"expires_at\": \"2027-03-13T10:12:00Z\", \"total_brl\": 560.00}"}],
            "isError": false}}

Notice what is not an argument: the fan's identity. The MCP server takes the user from the authenticated session (the OAuth access token the host obtained on the fan's behalf, as in Identity) — never from a field the model fills in, or a prompt injection could hold seats for someone else.

The registry around MCP

MCP standardizes the wire; the Tool Registry component is the governance around it:

For communication between agents owned by different teams or companies, separate protocols are emerging (for example Agent2Agent, A2A). The distinction to keep: MCP connects an agent to tools and data; agent-to-agent protocols connect an agent to another agent.

Multi-agent orchestration

When one agent has too many tools, too long a prompt, or tasks that can run in parallel, the work is split across several agents.

sequenceDiagram
    actor F as Fan
    participant S as Supervisor agent
    participant T as Tickets agent
    participant L as Lodging agent
    F->>S: "festival weekend in Rio, budget R$2,000"
    par in parallel
        S->>T: find 2 passes ≤ R$900
        S->>L: find lodging near the venue ≤ R$1,100
    end
    T-->>S: 2 options (hold not yet placed)
    L-->>S: 3 options
    S-->>F: combined proposal: confirm?

Multi-agent systems trade money and latency for modularity. Each agent has its own context (more tokens in total), every hand-off loses information unless the state is passed explicitly, and failures become harder to trace. Pass explicit, structured artifacts between agents (a list of candidate seats with IDs), not whole transcripts, and trace the full tree of calls (Observability).

When NOT to use agents

BoxOffice: the trace of one task

Parallel tool calls saved one full round (≈ 2 s). Had the model asked for a 40-seat hold, step 3 would have returned {"error": "deny: invalid arguments: List should have at most 6 items after validation, not 40"} (the guardrail check) and the model would have explained the limit to the fan.

How it connects

flowchart LR
    WF{"steps known<br/>in advance?"} -->|"yes"| FLOW["workflow<br/>(code decides)"]
    WF -->|"no"| LOOP["agent loop<br/>(model decides)"]
    LOOP --> BUD["budgets: steps,<br/>tokens, time"]
    LOOP --> DUR["durable execution:<br/>state after every step"]
    LOOP --> MATH["0.95^10 ≈ 60 %<br/>→ Estimation availability math"]
    LOOP --> TOOLS["tool calls via MCP<br/>→ tool registry"]
    TOOLS --> AUTH["authorized as the user<br/>→ Network & Security identity"]
    TOOLS --> IDEM["idempotency keys, compensation<br/>→ Messaging sagas"]
    LOOP --> CTX["context<br/>→ RAG & Memory"]
    TOOLS --> HITL["confirmation for risky actions<br/>→ Safety mesh"]

Palette components: orchestrator (runs the loop and owns the budgets) · toolreg (MCP servers, permissions, risk tiers) · llmgw (every model call) · safety (approval of risky actions) · auth (the user's identity on each tool call) · app (the same catalog and inventory services the web front end calls). Scenarios: ai-assistant. Continues in: Messaging — patterns (sagas and compensation), Network & Security — identity, Estimation — availability math, Compute — integrations (idempotency keys), RAG & Memory, Safety, Evals & Observability.