What is tool calling?
Tool calling is the mechanism by which a language model requests an action from software outside itself: it emits a structured request naming a function and its arguments, an application runs that function, and the result returns to the model as new context for its next turn.
A model cannot execute anything on its own. It produces the request; the surrounding application decides whether to run it, with which credentials, and against which account. That boundary is why tool calling is a permissions design problem long before it is a modeling one.
How tool calling works
Tool calling runs as a five-stage loop, and every stage fails in its own way.
Declaration comes first: the application sends the model a list of available tools, each with a name, a description of when to use it, and a typed parameter schema. Selection is the model reading the request and deciding which tool, if any, applies. Invocation is the model emitting arguments that conform to that schema, which is why tool calling depends entirely on structured output: prose cannot be executed. Execution happens in your code, where the request is validated, authorized, and run against a real system. Observation feeds the return value back into the conversation, and the model either answers or calls again.
That repetition is what the ReAct agent pattern formalized, alternating written reasoning with calls so each observation shapes the next decision. An AI agent framework supplies everything outside the model: the tool registry, the retry policy, the call budget, and the state that survives between turns.
Types of tool calling
Read tools: Calls that fetch state without changing it, such as order lookups, account balances, or subscription status, and are safe to retry.
Write tools: Calls that change a system of record, issuing a refund or cancelling a booking, which need idempotency keys and an audit line.
Parallel calls: The model emits several independent requests in one turn, cutting latency when an answer needs three records that do not depend on each other.
Chained calls: Each call's output becomes the next call's input, resolving an email address to a customer ID before fetching that customer's open cases.
Gated calls: The model proposes a call and a human or a rules engine approves it before execution, standard for anything above a value threshold.
Tool calling vs function calling vs retrieval vs orchestration
These four terms get used interchangeably in architecture documents, and the overlap is real, because one agent turn can involve all of them. Function calling names the same capability as tool calling, inherited from the model APIs that first shipped it. Retrieval fetches passages of text to inform an answer and changes nothing in any system. Orchestration decides which agent or step runs next, sitting one layer above any individual call. Tool calling is the request itself: one function, one set of arguments, one returned result.
What it does | Where it runs | What it returns | Fails when | Choose it when | |
|---|---|---|---|---|---|
Tool calling | Lets a model request a named function with typed arguments | Model proposes, your application executes | A real result from a live system | Arguments are malformed or unauthorized | The answer needs live data or a state change |
Function calling | Names the same capability in most model API documentation | Inside the model API contract | A JSON object naming a function and its arguments | Teams assume the API itself executes the call | You are reading provider documentation |
Retrieval | Finds text passages relevant to the question | In a search or vector layer before generation | Source text to ground an answer | The answer lives in a record, not a document | The answer already exists in written content |
Orchestration | Decides which agent or step runs next | Above individual calls, across turns and agents | Routing decisions and shared state | The calls underneath it are unreliable | One task spans several agents or systems |
If the answer lives in a document, retrieval is enough. If it lives in a system of record, or the customer wants something changed, you need tool calling, and orchestration only becomes worth its cost once several tool-using agents share one task.
Why tool calling matters for customer experience
Support automation without tool calling can only describe. Ask where an order is and a retrieval-only assistant quotes the shipping policy; ask for a refund and it explains the refund window. The customer wanted a state change and received a paragraph.
It also removes a class of wrong answers. When a model has no way to read the live record, it answers from training data or from whatever a retrieved article implied, which is one common route to a hallucination that sounds authoritative and quotes a balance nobody has.
The tradeoff is blast radius. A read-only assistant that is wrong wastes a customer's time, while a write-enabled agent that is wrong moves money, cancels a booking, or sends the wrong person a confirmation. Every tool you add raises the cost of a single bad decision.
How is tool calling measured?
Tool calling reliability decomposes into four numbers, and teams that report only the last one have nothing to act on.
Tool selection accuracy is the share of turns where the agent chose the correct tool, including the turns where the correct choice was to call nothing. Argument validity is the share of emitted calls that parse against the schema and reference entities that exist: well-formed JSON carrying a fabricated order ID still fails. Execution success is the share of validated calls the downstream API completes, which measures your integrations more than the model. Task completion is the end-to-end share of conversations that reached the intended state change.
Public comparison is harder than internal measurement. The Stanford HAI AI Index reported in 2025 on agentic benchmarks such as RE-Bench, where models and human experts attempt identical engineering tasks under time budgets ranging from 2 to 32 hours, with AI systems ahead at the short budget and human experts ahead at the long one.
How AI agents change tool calling
Early integrations wired one model response to one function and stopped there. An agent decides, on every turn, whether to call anything, which tool applies, and what the returned data means, then repeats until the goal is reached or a limit stops it. Parallel calls pull the order, the payment, and the shipping record at once, and a failed call becomes an observation the agent has to reason about.
That autonomy moves the engineering work from writing the flow to constraining it: call budgets per conversation, idempotency keys so a retried refund never double-pays, timeouts that degrade to a human handoff, and approval gates on destructive tools. The visible consequence in support is scope. Action-taking triage platforms can close a ticket end to end, where earlier systems could only classify it and route it onward.
What to look for in a tool calling implementation
Coverage is the first axis: which systems the agent can reach, and whether writes are included or the tool list stops at lookups. Integration surface is second: typed schemas, per-tool authentication scoped to the acting customer, and a sandbox where a call can be dry-run before it touches production data.
Governance decides who may add a tool and who signs off on one that moves money. Every call needs a log line carrying the arguments, the identity it acted under, and the result, because that log is the only reconstruction of what the agent actually did. Regulated buyers ask how that evidence is produced and retained, which is where SOC 2 AI support enters procurement conversations, and healthcare and financial teams raise the same question about their own regimes before any write tool is switched on.
The constraint most teams underestimate is schema drift. A downstream API renames a field, the call still returns a success code, and the agent acts confidently on a null.
Tool calling and agent orchestration
One agent calling five tools is a program you can reason about. AI agent orchestration is what happens when several agents, each with its own tool set, share state and hand work between them, and it inherits every failure mode of the calls underneath it.
Tool calling also crosses channels. An outbound voice AI agent placing a payment reminder has to look up the balance and record a promise-to-pay while the customer is still on the line, which imposes a latency budget that a chat agent never feels.
What does tool calling mean in plain terms?
Think of a language model as a very well-read colleague who cannot log into anything. Tool calling is giving that colleague a phone extension and a stack of forms, plus one rule: the form has to be filled in exactly the way the back office accepts, or it comes straight back.
Without those forms, the colleague answers every question from memory and general knowledge. Ask about your specific account and they will describe how accounts usually work, fluently, and never mention that they had no way to look.
The tradeoff arrives with the forms. Mistakes stop being words. A wrong sentence is embarrassing; a wrong form cancels a subscription. That is why sensible teams start with the forms that only read information and add the ones that change things one at a time, with someone signing off in between.
Common tool calling mistakes
Writing tool descriptions as API documentation is the first. The model selects a tool by reading that description, so vague wording or two tools whose descriptions overlap produce a selection bug that looks like a reasoning failure.
Treating the model's request as authorization is the second and most expensive. A model can be argued into proposing a call it should never make, so the decision about whether this customer may refund this order belongs in the executing code, checked against the session identity.
Returning raw payloads is the third. Passing a two-hundred-field JSON response back into context buries the one field that mattered, inflates cost, and invites the model to interpret data it was never meant to see.
The fourth is having no vocabulary for failure. Tools that return empty strings or free-text errors leave the model guessing, which is where guardrails in agentic support earn their keep: typed errors, retry limits, and a defined path to a human.
What is the difference between tool calling and function calling?
Tool calling and function calling describe the same mechanism under two names. Function calling was the original label used in model API documentation for emitting a structured request naming a function and its arguments. Tool calling became the broader term once agents began invoking retrieval, code execution, and other capabilities that are awkward to call functions.
What is tool calling in an LLM?
Tool calling in an LLM is the model's ability to produce a machine-readable request for external software rather than only text for a person. The application receives that request, validates it, runs the underlying function, and returns the result to the model, which uses it to continue the conversation or issue another call.
Tool calling vs RAG: which does an AI agent need?
Tool calling and RAG answer different halves of a question. Retrieval finds written content such as policies and help articles, so it handles questions whose answer already exists in documents. Tool calling reads live records and changes them. A support agent handling "where is my order" needs both: policy text and the current shipment status.
How do you keep tool calling secure?
Tool calling security lives in the executing application. Scope each tool's credentials to the authenticated customer, check authorization in code before running anything, keep destructive operations behind approval gates or value thresholds, and log every call with its arguments, identity, and outcome. Treat any argument the model produces as untrusted input requiring validation.
Can tool calling take actions in a CRM or ticketing system?
Tool calling can write to a CRM, ticketing system, billing platform, or order management system whenever those systems expose an API and the integration grants write permission. Common actions include updating a case status, issuing a refund, rescheduling a delivery, and creating a follow-up task. Idempotency keys prevent a retry from duplicating the action.
Why does an AI agent call the wrong tool?
Wrong tool selection usually traces to the tool definitions. Descriptions that overlap, omit the conditions for use, or read like internal API notes give the model insufficient signal to choose between them. Too many registered tools worsens it. Narrowing the tool list per conversation and rewriting descriptions around user intent fixes most cases.