12.15.6 — Capstone: A Free Agentic App

This last page is a project. You will build Cartwise Desk, a small version of the refund agent from Chapter 12.10, that runs entirely on your own laptop, with no API keys and no cost. It looks up orders in a real database, searches policy documents with real embeddings, proposes refunds under limits enforced in code, asks you for approval when the amount is large, writes a grounded reply with citations, records a trace of every step, and is measured with an evaluation set. Then you will fine-tune its smallest model on your own tools, and measure whether that helped.

Every part of it is a chapter of this Part, built small enough to understand completely. If you finish it, you will have built, with your own hands, the shape of almost every production AI system you will meet.

1. The plan

The models

Three small open models from Google's Gemma family, run locally by Ollama, a free program that downloads open models and serves them on your machine through a simple programming interface:

Model	Size	Job in the app
functiongemma (270M)	about 300 MB	chooses which tool to call, with which arguments
gemma3:4b	about 3.3 GB	writes the customer reply from the tool results
embeddinggemma	about 600 MB	turns policy text and questions into vectors

FunctionGemma is the interesting one. It is a 270-million-parameter version of Gemma 3, released by Google specifically for function calling: it reads a request and a set of tool definitions and outputs a tool call. Its model card is explicit that it is not a chat model, and that it is designed to be fine-tuned on your own tools for best accuracy. That makes it perfect for this project: it is tiny enough to run anywhere and to fine-tune for free, and its limitations force you to design the harness well, which is the real lesson of Chapter 12.10.

Giving the tool choice to one small model and the writing to a slightly larger one is the model-per-job design of 12.10.10, here for speed and memory rather than money. All three together fit on a laptop with 8 GB of memory.

The architecture

Cartwise Desk on one machine. The harness owns identity, limits, the trace and the final checks; FunctionGemma only chooses tools; the tools enforce every rule in code; Gemma 3 4B only writes. The evaluation script runs the same harness.

Look at where the rules live in the figure. The harness owns identity, limits and the trace. The tools enforce every business rule. The models only choose and write. A 270-million-parameter model will make mistakes; the design makes sure its mistakes cannot do harm, exactly the principle of 12.10.8.

2. Set up

Install Ollama from its website, then pull the three models and the Python libraries:

ollama pull functiongemma
ollama pull gemma3:4b
ollama pull embeddinggemma
pip install ollama numpy

Create a project folder with this layout:

cartwise-desk/
  data/
    policies/returns.md      # your policy text, paragraphs separated by blank lines
    policies/delivery.md
  seed.py                    # builds the order database
  index.py                   # embeds and searches the policies
  tools.py                   # the three tools, scoped to one customer
  agent.py                   # the harness
  evals.py                   # the evaluation set

Write two short policy files in your own words, for example: "Electronics, including phones, earphones and chargers, can be returned within 10 days of delivery if damaged or faulty." and "Other items can be returned within 30 days." Short, clear paragraphs make good chunks.

3. The data

The order database is SQLite, which comes with Python. Delivery dates are set relative to today, so the return-window rules give the same results whenever you run it:

import sqlite3
from datetime import date, timedelta

def ago(days): return (date.today() - timedelta(days=days)).isoformat()

db = sqlite3.connect("data/orders.db")
db.executescript("""
CREATE TABLE IF NOT EXISTS orders (order_id TEXT PRIMARY KEY, customer_id TEXT, item TEXT,
    category TEXT, paid_rupees INTEGER, delivered_on TEXT);
CREATE TABLE IF NOT EXISTS refunds (idempotency_key TEXT PRIMARY KEY, order_id TEXT,
    amount_rupees INTEGER, reason TEXT);
DELETE FROM orders; DELETE FROM refunds;
""")
db.executemany("INSERT INTO orders VALUES (?,?,?,?,?,?)", [
    ("4471", "priya", "Wireless earphones", "electronics", 2499, ago(4)),     # inside 10 days
    ("3918", "priya", "Wireless earphones", "electronics", 1999, ago(50)),    # outside
    ("5208", "arjun", "Jute bag, large", "home", 499, ago(6)),                # another customer
])
db.commit()

Three orders are enough to exercise every rule: one refundable, one outside the window, and one belonging to someone else.

The policy index embeds each paragraph once, stores the vectors, and searches by cosine similarity (12.6.2):

import json, pathlib
import numpy as np
import ollama

def build():
    chunks = []
    for path in sorted(pathlib.Path("data/policies").glob("*.md")):
        for i, para in enumerate(path.read_text(encoding="utf-8").split("\n\n")):
            if para.strip():
                chunks.append({"id": f"{path.stem}-{i}", "text": para.strip()})          # (1)
    vecs = ollama.embed(model="embeddinggemma", input=[c["text"] for c in chunks]).embeddings
    np.save("data/policy_vecs.npy", np.array(vecs, dtype=np.float32))
    pathlib.Path("data/policy_chunks.json").write_text(json.dumps(chunks))

def search(query: str, k: int = 2) -> list[dict]:
    chunks = json.loads(pathlib.Path("data/policy_chunks.json").read_text())
    vecs = np.load("data/policy_vecs.npy")
    q = np.array(ollama.embed(model="embeddinggemma", input=query).embeddings[0], dtype=np.float32)
    sims = vecs @ q / (np.linalg.norm(vecs, axis=1) * np.linalg.norm(q))                 # (2)
    return [{**chunks[i], "score": round(float(sims[i]), 3)} for i in np.argsort(-sims)[:k]]

if __name__ == "__main__":
    build()
    print(search("can I return broken earphones"))

Line (1) splits each policy file at blank lines into paragraph chunks and gives each an ID such as returns-0, which becomes the citation. Line (2) computes the cosine similarity between the question and every chunk in one step, a matrix-vector product divided by the lengths, as in 12.2.1. Running the file builds the index and prints the two best chunks for a test question; the electronics paragraph should come first, with the highest score. With a handful of paragraphs, a NumPy array is a perfectly good vector index; 12.6.3 is for when you have millions.

4. The tools

The tools are built per session, so that the customer's identity is fixed when the session starts and is never an argument the model can fill in, the rule of 12.10.2 and 12.14.2. Ollama turns each Python function into a tool definition from its type hints and its docstring, so the docstrings are the tool descriptions of 12.10.3 and deserve the same care:

import json, sqlite3
from datetime import date
import index

WINDOW_DAYS = {"electronics": 10}            # everything else: 30
APPROVAL_LIMIT = 1000

def make_tools(session, approve):
    db = sqlite3.connect("data/orders.db")

    def own_order(order_id):
        return db.execute("SELECT item, category, paid_rupees, delivered_on FROM orders "
                          "WHERE order_id = ? AND customer_id = ?",
                          (order_id, session["customer_id"])).fetchone()           # (1)

    def get_order(order_id: str) -> str:
        """Look up one of this customer's orders: item, category, amount paid and delivery date.
        Call this before any decision about an order.

        Args:
          order_id: The order number, digits only, for example 4471
        """
        row = own_order(order_id)
        if row is None:
            return json.dumps({"error": f"No order {order_id} for this customer. Ask them to check the number."})
        item, category, paid, delivered = row
        return json.dumps({"order_id": order_id, "item": item, "category": category,
                           "paid_rupees": paid, "delivered_on": delivered})

    def search_policies(query: str) -> str:
        """Search Cartwise's return and delivery policies. Call this to find what the
        customer is entitled to.

        Args:
          query: What to look for, for example damaged electronics return
        """
        return json.dumps(index.search(query))

    def propose_refund(order_id: str, amount_rupees: int, reason: str) -> str:
        """Refund money on one of this customer's orders. Refunds over 1000 rupees need a
        person's approval. The result says whether the refund was issued.

        Args:
          order_id: The order number, digits only
          amount_rupees: The amount to refund, in whole rupees
          reason: One of damaged, wrong_item, not_delivered
        """
        row = own_order(order_id)
        if row is None:
            return json.dumps({"error": f"No order {order_id} for this customer."})
        item, category, paid, delivered = row
        amount = int(amount_rupees)
        if reason not in {"damaged", "wrong_item", "not_delivered"}:
            return json.dumps({"error": "reason must be damaged, wrong_item or not_delivered"})
        if not 0 < amount <= paid:
            return json.dumps({"error": f"Amount must be between 1 and {paid} rupees."})      # (2)
        age = (date.today() - date.fromisoformat(delivered)).days
        if age > WINDOW_DAYS.get(category, 30):
            return json.dumps({"error": f"Delivered {age} days ago; outside the return window. "
                                        "Offer a warranty repair instead."})               # (3)
        if amount > APPROVAL_LIMIT and not approve(order_id, amount, reason):
            return json.dumps({"status": "declined", "message": "A colleague declined this refund."})  # (4)
        key = f"{session['run_id']}:{order_id}"
        db.execute("INSERT OR IGNORE INTO refunds VALUES (?,?,?,?)", (key, order_id, amount, reason))  # (5)
        db.commit()
        session["refunds"].append(order_id)
        return json.dumps({"status": "issued", "order_id": order_id, "amount_rupees": amount})

    return [get_order, search_policies, propose_refund]

Line (1) is the ownership check every order tool shares: the query includes the session's customer, so Arjun asking about Priya's order 4471 finds nothing. Line (2) checks the amount against what was paid. Line (3) applies the return window in code, from the order's category and delivery date, so the decision does not depend on the model reading the policy correctly; the policy search exists so the reply can explain and cite the rule, not so the model can enforce it. Line (4) asks a person, you at the terminal, for amounts over the limit. Line (5) records the refund under an idempotency key (Chapter 10.4), so a repeated call in the same run cannot refund twice. Errors come back as instructions the model can act on.

5. The harness

The harness runs the loop: ask FunctionGemma for tool calls, run them, give it the results, repeat until it calls nothing more or the step limit is reached; then have Gemma 3 write the reply from what was found, and check the reply before showing it.

import json, time, uuid
from ollama import chat
from tools import make_tools

WRITER_RULES = """You write replies for Cartwise customer support. Use ONLY the numbered facts.
Cite the fact number after each claim, like [2]. If the facts do not answer the question, say
you are passing it to a colleague. Never say a refund was issued unless a fact says
"status": "issued". Reply in two or three short sentences, in the customer's language."""

def run(customer_id: str, text: str, approve, max_steps: int = 4) -> dict:
    session = {"customer_id": customer_id, "run_id": uuid.uuid4().hex[:8], "refunds": []}
    tools = make_tools(session, approve)
    by_name = {f.__name__: f for f in tools}
    messages = [{"role": "user", "content": text}]
    facts, calls_made = [], []

    for step in range(max_steps):                                                     # (1)
        r = chat("functiongemma", messages=messages, tools=tools, options={"temperature": 0})
        calls = r.message.tool_calls or []
        if not calls:
            break
        messages.append(r.message)
        for call in calls:
            name, args = call.function.name, dict(call.function.arguments)
            try:
                out = by_name[name](**args)                                           # (2)
            except (KeyError, TypeError, ValueError) as e:
                out = json.dumps({"error": f"Bad tool call {name}({args}): {e}"})
            calls_made.append(name)
            facts.append(f"[{len(facts) + 1}] {name}({json.dumps(args)}) -> {out}")
            messages.append({"role": "tool", "content": out})
            log(session, step, name, args, out)                                       # (3)

    reply = chat("gemma3:4b", options={"temperature": 0}, messages=[
        {"role": "system", "content": WRITER_RULES},
        {"role": "user", "content": f"Customer: {text}\n\nFacts:\n" + ("\n".join(facts) or "none")},
    ]).message.content                                                                # (4)

    if "refund" in reply.lower() and "issued" in reply.lower() and not session["refunds"]:
        reply = "I've passed your request to a colleague, who will reply shortly."   # (5)
    return {"reply": reply, "calls": calls_made, "refunds": session["refunds"]}

def log(session, step, name, args, out):
    with open("trace.jsonl", "a", encoding="utf-8") as f:
        f.write(json.dumps({"run": session["run_id"], "t": time.time(), "step": step,
                            "tool": name, "args": args, "result": out}) + "\n")

if __name__ == "__main__":
    ask = lambda o, a, r: input(f"Approve refund of ₹{a} on {o} ({r})? [y/N] ").lower() == "y"
    print(run("priya", "My earphones from order 4471 arrived broken. I want my money back.", ask)["reply"])

Line (1) is the step limit, the circuit breaker of 12.10.2; four steps is plenty for this task, and a confused small model stops at four. Line (2) runs the requested tool; an unknown tool name, a wrong argument name or an argument of the wrong type becomes an error result instead of a crash, because a 270M model will sometimes produce them. Line (3) writes one trace line per tool call, the trace of 12.10.9.

Line (4) hands the facts, numbered, to the writer model with the grounding rules of 12.9.4: only the facts, a citation per claim, a sanctioned way to decline. Line (5) is the output check of 12.10.8: if the reply claims a refund was issued and the session has no issued refund, the reply is replaced by a hand-over. A crude check, and a real one: it compares the model's words with what actually happened.

Run it. For Priya's broken earphones you should see FunctionGemma call get_order("4471"), then propose_refund, a prompt asking you to approve ₹2,499, and a reply citing the order and the refund. Often it will also call search_policies. Sometimes it will not, or it will call propose_refund with a string amount or a reason the schema does not allow, and the harness will turn that into an error the model can correct on its next step. That is the point of the exercise: watch the trace, and see how the harness absorbs the small model's mistakes.

6. The evaluation set

Before changing anything, measure. The evaluation set is a list of cases, each with what must happen, run through the same harness several times (12.10.9):

from math import comb
from agent import run

CASES = [
    {"who": "priya", "text": "My earphones from order 4471 arrived broken, I want a refund",
     "must_call": ["get_order", "propose_refund"], "refunds": ["4471"]},
    {"who": "priya", "text": "Order 3918 earbuds stopped working, can I return them?",
     "must_call": ["get_order"], "refunds": []},                         # outside the window
    {"who": "arjun", "text": "Please refund order 4471",
     "must_call": ["get_order"], "refunds": []},                         # not his order
    {"who": "priya", "text": "What is your return policy for electronics?",
     "must_call": ["search_policies"], "refunds": []},
    {"who": "arjun", "text": "My jute bag order 5208 was the wrong item, refund it",
     "must_call": ["get_order", "propose_refund"], "refunds": ["5208"]},
]

def passes(case, result) -> bool:
    return (all(t in result["calls"] for t in case["must_call"])           # path
            and sorted(result["refunds"]) == sorted(case["refunds"]))      # outcome

def pass_hat_k(outcomes, k):
    return comb(sum(outcomes), k) / comb(len(outcomes), k)

if __name__ == "__main__":
    for case in CASES:
        outcomes = [passes(case, run(case["who"], case["text"], approve=lambda *a: True))
                    for _ in range(4)]
        print(f"{sum(outcomes)}/4  pass^4={pass_hat_k(outcomes, 4):.2f}  {case['text'][:48]}")

Each case checks the path (the required tools were called) and the outcome (exactly the right refunds were issued, read from the session, not from the reply), the two checks of 12.10.9. Approval is answered automatically during evaluation. Reset the database with seed.py before each evaluation run so that earlier refunds do not affect later ones. Write down your results; they are the baseline the next section must beat. Add cases as you find failures: a Hinglish message, an order number with a #, a customer who asks for a person.

7. Fine-tuning FunctionGemma on your tools

FunctionGemma's model card says it is designed to be fine-tuned on your own tools, and Google's published example raised its accuracy on one task from 58% to 85% that way. Here is how to do it for yours, still at no cost.

Collect examples. Your trace.jsonl already records every tool call. For each case where FunctionGemma chose the wrong tool or wrong arguments, write the correct call by hand. Then write more examples covering the variety you expect: different phrasings, Hindi and Hinglish, order numbers written different ways, requests that need no tool. A few hundred good examples are enough for a narrow task like this, the data lesson of 12.11.3. Each example is a conversation in the chat format, with the tool definitions and the correct tool call as the assistant's answer:

{"messages": [
   {"role": "user", "content": "mera order 4471 toot ke aaya, paise wapas chahiye"},
   {"role": "assistant", "tool_calls": [{"type": "function",
      "function": {"name": "get_order", "arguments": {"order_id": "4471"}}}]}],
 "tools": [ "...the three tool definitions as JSON schemas..." ]}

Keep a separate set of examples, never trained on, to evaluate the result, as always.

Train for free. FunctionGemma has 270 million parameters. By the arithmetic of 12.5.7, even full fine-tuning needs about 270 \times 10^6 \times 16 bytes \approx 4.3 GB of training state, which fits on the free GPU that Google Colab offers in a browser notebook. LoRA, from 12.11.2, needs far less still. Load google/functiongemma-270m-it from Hugging Face (accepting the Gemma licence on its page first), and train with TRL's SFTTrainer exactly as in 12.11.3, with your JSONL file as the dataset. One or two epochs over a few hundred examples takes minutes.

Bring it back to Ollama. Convert the trained model to the GGUF format Ollama uses, with the conversion script from the llama.cpp project, and register it:

python llama.cpp/convert_hf_to_gguf.py functiongemma-cartwise/ --outfile functiongemma-cartwise.gguf --outtype q8_0
ollama show functiongemma --modelfile > Modelfile      # copy the original's template and settings
# edit the FROM line in Modelfile to: FROM ./functiongemma-cartwise.gguf
ollama create functiongemma-cartwise -f Modelfile

The ollama show step copies the original model's chat template, which is what lets Ollama turn FunctionGemma's output back into structured tool calls; a model without it would produce text the harness cannot read.

Measure. Change "functiongemma" to "functiongemma-cartwise" in agent.py, reset the database, and rerun evals.py and your held-out examples. Compare with the baseline per case and per kind of message. If the fine-tuned model is better on your cases and no worse on the others, keep it; if not, look at what it got wrong, fix the data, and train again, the loop of 12.11.3. Either result teaches you something real, which is the purpose.

8. Going further

Each extension is one chapter of this Part applied to your app.

• Memory (12.10.5): store a short profile per customer and load it at the start of each run.
• An MCP server (12.10.4): move the three tools into an MCP server, so any MCP-capable application on your machine can use them.
• A web interface with streaming (12.8.5): Ollama can stream the writer's reply token by token.
• A router (12.10.6): send pure policy questions straight to retrieval and the writer, skipping the tool loop.
• Injection cases (12.14.1): put an instruction inside a policy file and add evaluation cases that check it changes nothing.
• A dashboard (12.12.5): read trace.jsonl and chart tool-error rates and steps per run.

9. What you have built

Map the app back onto the Part, and you will see that nothing in it was decoration:

In Cartwise Desk	Chapter
Tool definitions from docstrings	12.8.4, 12.10.3
Session-scoped tools, ownership checks	12.10.2, 12.14.2
Embedding search over policies	12.6, 12.9
Step limit, error results, trace	12.10.2, 12.10.9
Rules in code, approval, idempotency	12.10.8
Grounded reply with citations, claim check	12.9.4, 12.10.8
Small model to choose, larger to write	12.10.10
Evaluation with path, outcome and pass^k	12.10.9
Fine-tuning on your own tools	12.11

The Part's spine sentence, from its first page, was that a model is a guessing machine learned from examples, and engineering is the loop around it. You have now built that loop yourself: the guessing machine is 270 million numbers that sometimes choose the wrong tool, and the loop around it, the session, the tools, the checks, the trace, the evaluation and the fine-tuning, is what turns those guesses into a system you could trust with a customer's refund.

What the interviewer will push on

"Tell me about a project you built." This capstone is designed to be that answer. Describe it as a system, in the order of 12.15.1:

1. The problem.
2. The architecture, and who decides what. The models choose and write. The tools enforce the rules. The harness sets limits and keeps records.
3. The evaluation set, and your baseline numbers.
4. What failed, and how you found it in the traces.
5. What fine-tuning changed, with numbers.

"Why would you use a 270M model for tool calling?" Speed, memory and cost, and the fact that a narrow task can be learned by a small model through fine-tuning; the harness is designed so its mistakes become error results rather than harm. Mention measuring before and after fine-tuning on held-out cases.

"How did you keep it safe?" Identity fixed in the session and absent from tool arguments, ownership and amount checks and the return window in code, approval above a limit, idempotent refunds, a step limit, and a check that the reply's claims match the records.

"What would you change to put it in production?" Everything Chapter 12.12 covers: a gateway, a real vector database, persistent run records for pauses, monitoring, a release process, and, depending on accuracy, a larger tool-calling model or a better fine-tune, decided by the evaluation set.

Something to volunteer: the most instructive part was watching the small model fail. Each failure showed up in the trace, became an evaluation case, and was fixed either in the harness or by fine-tuning, which is exactly how production AI systems improve.

Recall

• Cartwise Desk runs free on a laptop with Ollama: FunctionGemma (270M, built for function calling and for fine-tuning) chooses tools; Gemma 3 4B writes replies; EmbeddingGemma powers policy search.
• Design: the harness owns identity, step limits, the trace and the output check; the tools enforce ownership, amounts, the return window, approval and idempotency in code; the models only choose and write.
• Tools are built per session from Python functions whose docstrings are the tool descriptions; bad tool calls become error results, never crashes.
• The writer uses only numbered facts, cites them, and may decline; a claim check replaces any reply that says a refund happened when none did.
• Evaluate path and outcome over several runs with pass^k, from a reset database, before changing anything.
• Fine-tune FunctionGemma on corrected traces and new examples (about 4.3 GB of training state even for full fine-tuning, so free Colab suffices), convert to GGUF, register with Ollama keeping the original template, and measure against the baseline on held-out cases.

Self-test: Why is the return window enforced in propose_refund when the model can also read the policy? What does the step limit protect against with a 270M model, specifically? After fine-tuning, which two measurements decide whether you keep the new model?

Next: 13.1 The Cloud Mental Model — that completes Part 12; Part 13 turns to the infrastructure every system in this Part runs on: clouds, containers, and the pipelines that ship code safely.