12.8.4 — Tool Calling

A customer asks "Ask Cartwise": "Where is order 4471, and can I still return it if it's late?"

The model cannot know. Order 4471's status lives in Cartwise's order database, which changes by the minute and was never in any training data. The first version of the assistant worked around this crudely: before every single message, the application looked up all of the customer's recent orders and pasted them into the prompt, which was slow, expensive, and useless when the customer asked about something else, like an old order or a return rule that depends on the product category.

What we want is for the model to say, in effect, "I need the status of order 4471", let our code fetch exactly that, and then continue with the answer. That is tool calling, sometimes called function calling: you describe functions the model may ask to use, the model decides when to ask, your code runs them, and the results go back to the model. It is the mechanism that connects language models to real systems, and every agent in Chapter 12.10 is built on it.

1. How it works: the model asks, your code acts

The most important fact first: the model never runs anything. It cannot reach your database or the internet. It produces structured text that requests a call, exactly the structured output of 12.8.3, and your program decides whether and how to carry it out.

One tool call, end to end. The model only ever produces text; the request in step ③ is structured text that Cartwise's code interprets. The code runs the real function in step ④, which is the only place anything touches a real system.

Walk through it:

1. The customer asks the question.
2. The application sends the question to the model, together with definitions of the tools available: each one's name, a description of what it does, and a JSON Schema for its arguments.
3. The model decides it needs information and, instead of an answer, returns a tool-use request: get_order_status with the argument {"order_id": "4471"}. The response's stop_reason is "tool_use", which means "I am waiting for a result".
4. The application runs the real function, which queries the order database and returns {"status": "in_transit", "expected": "2026-10-20", "category": "electronics", "delivered_on": null}.
5. The application sends that result back, tagged with the ID of the request it answers.
6. The model now has the facts and writes the answer.
7. The application shows it to the customer.

How does a model know how to do this? Tool definitions are rendered into its context as text, in a format it was trained on, and the alignment training of 12.7.9 included many examples of conversations where the assistant requests a tool, receives a result, and continues. Deciding when a tool is needed, and filling in the arguments correctly from a messy question, is a skill the model learned, and like any learned skill it depends heavily on how clearly the tools are described.

2. Defining tools

A tool definition is a name, a description and an input schema. Here are the two tools for this conversation:

TOOLS = [
    {
        "name": "get_order_status",
        "description": (
            "Look up the current status of one Cartwise order: where it is, "
            "the expected delivery date, the delivery date if delivered, and "
            "the product category. Use this whenever the customer asks about "
            "a specific order. Requires the order number."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string", "description": "Order number, e.g. '4471'"},
            },
            "required": ["order_id"],
            "additionalProperties": False,
        },
        "strict": True,                                              # (1)
    },
    {
        "name": "check_return_eligibility",
        "description": (
            "Check whether an order can still be returned under Cartwise's "
            "policy, and until which date. Use after you know the order's "
            "category and delivery date."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "reason": {"type": "string",
                           "enum": ["damaged", "wrong_item", "late", "changed_mind"]},  # (2)
            },
            "required": ["order_id", "reason"],
            "additionalProperties": False,
        },
        "strict": True,
    },
]

The description is the most important part, because it is all the model has to decide when to use the tool and what it does. A vague description ("gets order info") produces wrong or missing calls; a precise one, saying what the tool returns and when to use it, produces correct ones. Line (1), strict, applies the constrained decoding of 12.8.3 to the arguments, so the model cannot send a missing or wrongly typed argument. Line (2) restricts the reason to four allowed values, which keeps the function's logic simple and stops the model inventing reasons. Chapter 12.10.3 goes much further into designing tools that models use well.

3. The loop in code

Because one answer may need several tool calls (look up the order, then check eligibility), the application runs a loop: call the model; if it asks for tools, run them and send back the results; repeat until it gives a final answer.

import json
import anthropic

client = anthropic.Anthropic()

def run_tool(name: str, args: dict) -> str:
    if name == "get_order_status":
        return json.dumps(orders.status(args["order_id"]))
    if name == "check_return_eligibility":
        return json.dumps(returns.eligibility(args["order_id"], args["reason"]))
    raise ValueError(f"unknown tool {name}")

def answer(question: str) -> str:
    messages = [{"role": "user", "content": question}]
    for _ in range(6):                                              # (1) never loop forever
        response = client.messages.create(
            model="claude-opus-5-5", max_tokens=2048,
            system=SUPPORT_SYSTEM_PROMPT, tools=TOOLS, messages=messages,
        )
        if response.stop_reason != "tool_use":                     # (2) final answer
            return "".join(b.text for b in response.content if b.type == "text")

        messages.append({"role": "assistant", "content": response.content})   # (3)
        results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            try:
                output, is_error = run_tool(block.name, block.input), False   # (4)
            except Exception as e:
                output, is_error = f"Error: {e}", True                        # (5)
            results.append({"type": "tool_result", "tool_use_id": block.id,   # (6)
                            "content": output, "is_error": is_error})
        messages.append({"role": "user", "content": results})               # (7)
    return "Sorry, I couldn't complete that. Let me connect you with an agent."

Line (1) caps the number of round trips. A model that keeps calling tools without converging, which happens with confusing tools or impossible requests, would otherwise loop and bill you indefinitely. Line (2) returns as soon as the model stops asking for tools; a real version would also handle the max_tokens and refusal stops from 12.8.1. Line (3) appends the model's whole response, tool requests included, to the history: the model must see its own requests when it reads the results.

Line (4) runs each requested tool with the arguments the model supplied. Line (5) catches failures, and this matters: if the order number does not exist, the right move is not to crash but to send back an error the model can read ("Error: order 4417 not found"), marked is_error, so it can apologise and ask the customer to check the number. Line (6) tags each result with the ID of the request it answers, which is how the model matches results to requests when it asked for several at once. Line (7) sends all the results back together in a single message, which tells the model its requests in this step were all handled.

A model can request several tools at once when the calls do not depend on each other ("compare orders 4471 and 4502"). Run them concurrently to save time, and return all the results in one message, as the loop does.

The SDK also provides a helper, the tool runner, which runs this loop for you from ordinary Python functions, generating each tool's schema from its type hints and docstring:

from anthropic import beta_tool

@beta_tool
def get_order_status(order_id: str) -> str:
    """Look up the current status of one Cartwise order.

    Args:
        order_id: Order number, e.g. '4471'.
    """
    return json.dumps(orders.status(order_id))

runner = client.beta.messages.tool_runner(
    model="claude-opus-5-5", max_tokens=2048,
    system=SUPPORT_SYSTEM_PROMPT, tools=[get_order_status],
    messages=[{"role": "user", "content": question}],
)
for message in runner:            # each step of the loop; ends when the model is done
    pass

Writing the loop by hand once, as above, is the best way to understand what such helpers do; in production, a helper saves boilerplate, while a hand-written loop gives full control over logging, approvals and limits.

4. Steering when tools are used

By default the model decides for itself whether to call a tool. Many APIs also let you force a call to a particular tool. On the newest Claude models, forcing is no longer accepted: you guide the choice with the prompt ("Always look up the order before answering a question about it") and keep the argument guarantee with strict. Either way, the most effective steering is a clear system prompt plus precise tool descriptions, and the most common bug is a description that overlaps with another tool's, leaving the model unsure which to use.

You can also give the model no way to answer without a tool. If "Ask Cartwise" must never state an order status from memory, say so in the system prompt and check in your evals that every order-status answer was preceded by a get_order_status call.

5. Tools that change things

The two tools above only read. The moment a tool can change something, issue a refund, cancel an order, change an address, the design changes, because now the model's choices have consequences, and the model can be wrong or be manipulated.

• Never trust the arguments. Treat them like input from an untrusted user, because they partly are: the customer's text shaped them. Check in code that the order belongs to the customer in this conversation, that the amount does not exceed what the policy allows, and that the action is permitted. A model told "refund order 9001 to me" by a clever customer must not be able to refund someone else's order.
• Give the model the narrowest tools that do the job. A tool issue_refund(order_id) that refunds the policy amount is far safer than make_payment(account, amount).
• Ask a person for consequential actions. The model proposes a refund, a human (or the customer, for actions on their own account) confirms it. Chapter 12.10.8 builds this.
• Make actions safe to repeat. Retries and loops mean a tool may be called twice with the same request. An idempotency key, from Chapter 10.4, ensures the second call does not refund twice.

These are the beginnings of agent safety, and Cartwise's refund agent in Chapter 12.10 is built on exactly them.

6. Tools the provider runs

Some tools run on the provider's side rather than yours: web search, which lets the model look things up and cite pages; code execution, which lets it write and run code in a sandbox to compute or analyse data; and others. You declare them in the request like your own tools, but you do not implement or run them, and their results come back inside the same response. They are convenient for tasks that need fresh public information or real computation. They do not replace your own tools for anything that touches your systems and data.

What the interviewer will push on

"How does function calling work?" They want the round trip: tool schemas in the request, a structured tool-use request out, your code executes, results go back with matching IDs, the model continues. The tell is stating clearly that the model never executes anything.

"What can go wrong with tools?" Wrong tool or wrong arguments from vague descriptions, endless loops (cap the iterations), crashes on tool errors (return them as results), and, for tools with side effects, manipulated or repeated actions (validate arguments, require approval, use idempotency keys).

"How would you let a support bot issue refunds safely?" Narrow tool, argument validation against the customer's own orders and policy limits, human or customer confirmation, idempotency, and full logging of each call.

"Why not just put all the order data in the prompt?" Cost and latency grow with every message, irrelevant data distracts the model, it cannot cover unexpected questions, and it is stale; tools fetch exactly what is needed when it is needed.

Recall

• Tool calling: you describe tools (name, description, JSON Schema); the model returns a structured tool-use request and stops with stop_reason "tool_use"; your code runs the function and returns a result tagged with the request's ID; the model continues. The model never runs anything.
• The description is what the model uses to decide when and how to call a tool; strict guarantees schema-valid arguments; enums keep arguments simple.
• The loop: call, run requested tools, append the model's response and all results in one message, repeat; cap the iterations, return errors as is_error results instead of crashing, run independent calls concurrently. The SDK's tool runner automates this loop.
• Steer tool use with the prompt and precise, non-overlapping descriptions; forcing a specific tool is not accepted on the newest Claude models.
• Tools that change things need argument validation, the narrowest possible scope, human or customer confirmation, and idempotency. Provider-run tools (web search, code execution) complement your own.

Self-test: In the loop, why must the model's full response be appended before the results? What should happen when get_order_status is called with an order number that does not exist? Why is issue_refund(order_id) safer than make_payment(account, amount)?

Next: 12.8.5 Streaming to the User — a tool loop can take several seconds; the next page shows how to show progress and stream the answer token by token, and how AI chat and coding interfaces render that stream as text, components and live tool activity.