Tool-call evals: did the agent pick the right tool and the right arguments

In June 2024 the authors of the tau-bench benchmark read through the failed runs of the best agent in their paper. The agent was built on gpt-4o and worked as a customer service assistant for a simulated shop. Of 115 runs on the retail tasks, 40 failed, and the authors traced 36 of the 40 to the agent and 4 to faults in the instructions given to the simulated customer [1]. Their summary of the agent's failures is one sentence: the agent "usually makes the right type of tool call(s) but fills in one or more arguments incorrectly" [1].

That sentence describes a failure in one step of a long task: the right tool, sent a wrong value. A test that only reads the agent's final message can miss it, because the message may still say the job is done. A tool-call eval looks at the single step and asks whether the request the agent sent was the right request. This page sets out its five checks, which of them code can decide, and how to build a first set of cases.

What a tool call is and why it is the smallest thing to test

A large language model (LLM) produces text. To act on anything, it has to ask another program to do the work. A tool call is that request: the model writes the name of a tool, such as "look up order", and the values to send with it, such as the order number. Your software runs the tool and returns the result to the model. The Berkeley researchers who maintain a public test of this skill define it this way: "Function calling, also called tool use, refers to an LLM's ability to invoke external functions, APIs, or user-defined tools in response to user queries" [2]. An API (application programming interface) is the set of requests one program accepts from another.

Each tool comes with a written definition called a schema. The schema lists the tool's name, what it does, which values it accepts, the type of each value (a number, a date, a piece of text), and which values are required. The values sent with a call are called arguments.

An agent's run is a sequence of these requests, and the full record of one run is called a trace. Why testing an agent is different from testing one model answer explains how an error in one step affects the steps after it. The tool call is the smallest part of a run that can be right or wrong by itself, so it is the first place to test. The rest of the method is in the AI agent evals guide.

The five checks for one step

For a single step, a tool-call eval asks five questions in order.

Check	How it is decided	Example failure
Was a tool needed?	Code: compare "called" or "did not call" with the case	A customer asks for opening hours and the agent calls the refund tool
Was the right tool chosen?	Code: compare the tool name with the accepted names	The agent calls "cancel order" when the case expects "look up order"
Do the arguments fit the schema?	Code: validate the request against the schema	A quantity is sent as the text "2" where a number is required
Are the argument values correct?	Code for exact values, judgment for free text	Order number 48213 is sent and the customer's order is 48231
Did the agent handle a tool error?	Code for the next action, judgment for the wording	The tool answers "order not found" and the agent says the refund is done

The shop, the tools and the order numbers in the table are an invented illustration.

Check 1: was a tool needed at all

Some requests need no tool. An agent that calls one anyway spends money and may change a record that should have stayed as it was.

The Berkeley Function Calling Leaderboard, known as BFCL, is a leaderboard (a public ranking table) of models on tool use, and it tests this case directly. Its first version included what the authors call relevance detection: "we design scenarios where none of the provided functions are relevant and supposed to be invoked. We expect the model's output to be no function call" [3]. The later paper says the benchmark measures "the ability of models to abstain" [2], meaning to decline to call a tool. This page uses the benchmark's method and quotes no model scores.

Copy that design. For every tool your agent has, write at least one case where the request looks similar to that tool's purpose and the correct action is to call nothing. The check is one line of code: the case passes when the step contains no tool call.

Check 2: was the right tool chosen

When a tool is needed, the first thing to compare is its name. BFCL's checking procedure starts there: it "first extracts the function name and verifies that it is consistent with the one in possible answer" [3]. OpenAI's documentation lists "Did the agent pick the right tool?" among the questions its trace grading is meant to answer [4].

Write each case so that the accepted tools are a stated list, because some requests can be met correctly by more than one tool. Put tools with similar purposes into the same cases: an order lookup and a customer lookup are a pair an agent can confuse.

The sources used for this page give no verified limit on how many tools one agent can handle. The tau-bench retail setting gave its agent 15 tools, 7 that change data and 8 that only read it [1], which describes one test and sets no maximum. Measure it on your own agent: add the tools and re-run the selection cases.

Check 3: do the arguments fit the schema

After the name, BFCL checks the arguments against the tool's definition in two steps. It confirms that "all required parameters, as identified by the "required" attribute in the function documentation, are present in the model output", and it "ensures that only parameters exist in the function doc are used, flagging the model hallucination outputs" [3]. A hallucination here means a field or a value the model made up. The paper calls this an Abstract Syntax Tree method: the call is broken into its parts (the name and each argument) and the parts are compared one by one [2].

Model vendors now offer a setting that enforces the schema while the model writes. OpenAI's documentation says that setting strict to true "will ensure function calls reliably adhere to the function schema, instead of being best effort", and recommends always turning it on [5]. Anthropic's documentation says its strict setting "guarantees Claude's tool inputs match your JSON Schema", and that without it the model "might return incompatible types ("2" instead of 2) or omit required fields" [6]. JSON (JavaScript Object Notation) is a text format for structured data. Each statement is a company's description of its own product.

Keep the schema check in your eval with the setting on. OpenAI's documentation lists cases where output can still fail to match the schema: when the model refuses a request for safety reasons, and when a response stops at the maximum length and is incomplete [7]. It also says strict mode is disabled when a fine-tuned model (one given extra training on a customer's own examples) calls several functions in one turn [5]. The check costs one validation per step.

Check 4: are the argument values correct

A request can fit the schema and still be wrong. The strict settings constrain the shape of the arguments: the right fields with the right types. Whether the order number is this customer's order number is a separate question. That reading of the vendor documentation is ours.

The tau-bench paper measured this failure. Its authors counted tool calls that used identifiers (for a user, a product, an order or an item) that were absent from the test database. The gpt-4o agent made 0.46 such calls per retail task, and the two gpt-3.5-turbo agents made 2.08 and 6.34 [1]. Those are 2024 models on one benchmark, and they show that this failure can be counted.

Test values in three ways:

• Exact values. Where the case has one correct value (an identifier, an amount, a date), compare in code. Decide in advance whether formats such as "2026-09-30" and "30 September 2026" both count.
• Existence. For every identifier, check in code that it exists in the test data. This catches invented identifiers in cases that have no expected answer.
• Free text. For a search query or a message to a customer, many wordings are right. Use a person or a model grader working from written scoring rules, as described in the three kinds of LLM eval.

Check 5: what the agent does when a tool returns an error

A tool can fail: a record is missing, a service is busy, a value is rejected. Anthropic's guide to building agents says an agent needs to learn the real state of the system from its environment at each step, and names tool call results as one place that information comes from [8]. An error message is one of those results.

The sources for this page report no measured rate for this failure, so the example is an invented illustration. Take a support agent for an invented furniture shop whose order tool answers "order not found". Three next actions are acceptable: look up the order a different way, ask the customer to confirm the number, or stop and pass the conversation to a person. Three are failures: sending the identical call again, making up the contents of the order, or telling the customer the task succeeded.

To test this, replace the real tool with a version that returns a scripted error, which test environments for agents describes. The next action can be checked in code: the case fails if the following call is identical to the failed one, or if the run ends with a success message. An agent that repeats a failed call again and again is in a loop, and cost, time and step limits are the control that ends it.

Why these checks are cheap, and what they leave out

Four of the five checks are comparisons that code can make, with no model grader and no person, so they can run on every change. A model grader costs money on every decision, which the real price of an LLM judge sets out with public figures.

Passing every single-step check is weaker evidence than a passing task. The BFCL paper's own finding is that leading models "excel at singleturn calls" while memory and long multi-step reasoning "remain open challenges" [2]. Whole-task grading is covered in outcome evals vs trajectory evals, and the business overview is how to evaluate an AI agent before you trust it.

A tool-call eval judges a step where the case defines the right request. Anthropic's guide to agent evals warns that checking a fixed sequence of calls across a whole task gives tests that fail when an agent finds another valid route [9]. Use step checks where one answer is right, and grade the route only for rules that must always hold.

How to build a first tool-call set

1. List every tool and its schema, and mark the tools that change data.
2. For each tool, write cases of four types: a clear request for that tool, a request that belongs to a similar tool, a request that needs no tool, and a request followed by a scripted error.
3. Take the inputs from real requests where you have them. LangSmith's documentation describes an agent test example as holding "correct tool selection and proper argument formatting" [10], and Anthropic suggests starting an agent suite with 20 to 50 simple tasks drawn from real failures [9].
4. Write the accepted tools, the accepted values and the forbidden next actions into each case before running anything.
5. Run each case several times, for the reason agent reliability across repeated runs explains.
6. Report each check as its own pass rate, because wrong tool choice and wrong values need different fixes.

How Reveneau applies this

At Reveneau, all code is written by AI, and every change must pass a large eval suite that is written from the specification before the code exists. When the product is an agent, tool-call cases are part of that suite. We write the accepted tools, the accepted argument values and the forbidden next actions for each case from the specification, before the prompt and the tool definitions are written. The four checks that code can decide run on every change. The judged checks, such as whether a free-text argument says the right thing, are graded by Jev, TypeSafe AI's decision model, and on Reveneau's own suite the run is ten times faster than with its previous language-model grader.

We use AI instead of hiring more engineers, so a build takes a small team, and that saving goes into the client's price. Reveneau, as a company, takes responsibility for the whole project through production and after release. To have tool-call evals written for an agent you are planning, see AI development at Reveneau or contact us.