Tool Calling

When a Model Can Choose a Capability

Tool calling can look like a small feature in a model API: present some tools, receive a structured request, run an operation, and return the result.

The shape is familiar. The control problem is different.

An ordinary application reaches a capability because deterministic code, configuration, or a user flow selected that path. A tool-using application can ask a model to interpret a goal, choose among described capabilities, and propose the arguments.

The model does not directly execute the operation. Depending on the system, the application, a provider, or a protocol host may run it. In every case, an execution boundary still has to validate, authorize, execute, and observe the operation.

That makes tool calling a useful boundary concept. The wire shape resembles an API call, but the caller behaves more like a probabilistic participant in control flow.

This is Post 04 in The Abstraction Shift: How Software Keeps Moving Up.

In One Sentence

Tool calling is a model-mediated capability invocation in which a model may select a named operation and propose structured arguments, while an execution host and the surrounding capability boundary remain responsible for authorization, validation, side effects, and the outcome that is reported.

The important distinction is between proposal and execution.

The model can propose:

• which tool matches the user’s goal;
• which arguments appear to fit the available context;
• whether another operation may be needed after seeing a result;
• whether it can answer without invoking a tool.

The runtime must decide:

• whether the tool exists and is available to this caller;
• whether the arguments satisfy the schema and business rules;
• whether the identity is authorized for this operation;
• whether a person must approve it;
• whether the call may run within the budget and side-effect policy;
• what actually happened when the capability returns.

Confusing those two responsibilities is the fastest way to turn a convenient API feature into an unsafe control surface.

Why This Exists

Language models are useful for interpreting open-ended requests. Enterprise systems are designed to enforce precise operations. Tool calling connects those strengths.

A user does not naturally ask for get_order_status(order_id="A-1842"). They ask, “Where is my order?” A model can map the request to a capability and infer that the order identifier is needed. The application can then execute a known operation against an order system and return the result as context for the next model turn.

Without tools, a model can describe what an application might do, but it cannot reliably access current private data or cause a business operation to happen. Without a model in the loop, the path from a goal to a capability has to be selected by code, configuration, or user interaction.

Tool calling occupies the space between those extremes. It lets software expose a controlled capability surface while allowing the model to participate in the choice of the next operation.

The capability surface can include read operations, such as retrieving an order or searching a knowledge base. It can also include consequential operations, such as creating a ticket, issuing a refund, changing an address, or deploying a service. The wire shape may be similar in both cases. The risk and required controls are not.

What We Did Before

The lineage is familiar.

Procedure and function calls

Within one process, a caller invokes a function with a name and arguments. The compiler, type system, and program structure constrain which calls are possible. The developer chooses the path.

RPC and service APIs

Remote procedure call systems make a function-like operation available across a process or network boundary. A client sends a structured request to a service and receives a result or error. Serialization, transport, timeouts, retries, and versioning make the remote boundary explicit.

An API client still normally knows which operation to call because that decision is encoded in application logic, configuration, or user interface flow.

Plugin systems and extension points

Plugin architectures let a host discover or load capabilities supplied by other components. The host may use metadata, registration, or conventions to decide which extension to invoke.

This moved capability selection away from a single compiled program, but the selection logic was still usually deterministic. A plugin registry did not normally infer the user’s goal and construct a valid argument object from a description.

Workflow activities

Workflow engines expose tasks or activities that can be composed into a longer process. The engine can retry, compensate, schedule, and persist state across steps. The workflow definition remains the primary source of control flow.

Tool calling can sit inside a workflow, but if the model proposes the next activity, the workflow no longer fully determines the path. That is the point where the comparison becomes incomplete.

Tool calling keeps the contract and execution boundary, but makes the caller’s choice and argument construction probabilistic.

The Abstraction Shift: How Software Keeps Moving Up

The old abstraction was roughly:

developer-selected operation
        -> typed arguments
        -> service execution
        -> result

The model-mediated version is closer to:

goal + context
        -> model proposal
        -> runtime validation and policy
        -> capability execution
        -> result as new context
        -> answer, retry, clarification, or another call

The tool interface did not replace the service boundary. It introduced a participant on the caller side that can interpret meaning and select among operations.

That creates four changes worth separating.

The caller can choose semantically

The model can choose a tool because its description appears relevant to the goal. This is not the same as a router matching a fixed path or a developer branching on a known field. The decision is based on the model’s interpretation of the current context.

Arguments can be generated from incomplete language

The model may extract arguments from the conversation, combine them with retrieved context, ask for missing information, or make an uncertain inference. A schema can constrain shape and type, but it cannot by itself establish that an argument is correct, authorized, current, or safe.

Results become part of the next context

The return value is often added to the conversation so the model can answer, correct itself, call another tool, or stop. Once it enters model context, the result participates in the next decision.

This makes provenance, freshness, error representation, and output size part of the model interface. A result that is technically valid but semantically ambiguous can steer the next step incorrectly.

The loop can continue without a fixed workflow

The model may call one tool, inspect the result, and choose another. A single user request can become a sequence of operations whose length and shape vary with the evidence encountered along the way.

That is the connection between tool calling and agents. Tool calling is a capability boundary. An agent is a control loop that may use that boundary repeatedly.

What’s Actually New?

Tool calling is not a new way to execute code. Its newness comes from combining familiar parts around a different caller.

1. Capability descriptions become executable context

The tool description is written for a model as well as for a developer. Its name, description, input schema, output behavior, examples, side effects, and limitations influence whether the model selects it.

That makes documentation operational. A vague description can produce a wrong selection even when the underlying implementation is perfect. A misleading description can make a dangerous tool appear safe.

2. The model participates in dispatch

In conventional dispatch, the caller or a deterministic router identifies the operation. With tool calling, the model can select the operation from a set of candidates. The dispatch problem becomes partly semantic.

The runtime should still reject unknown or unavailable tools. It should also treat selection quality as something to evaluate, not assume.

3. The interface has multiple audiences

An API contract serves code. A tool contract may be consumed by code, a model, and a human-facing host or operator. The schema constrains the request, while the natural-language description helps the model decide when the capability applies and may help a person understand an approval prompt.

Those parts have different failure modes. A schema can be valid while its description encourages the wrong use. A clear description can still produce arguments that fail a business rule.

4. Errors can become recovery context

In a conventional client, an error often returns to code that has a known recovery path. In a model-mediated loop, the error may be represented as a tool result that the model can use to correct an argument, ask a question, choose another tool, or stop.

That flexibility can improve recovery. It can also create loops, expose details that should remain internal, or teach the model to retry an operation that should not be retried.

5. Capability exposure becomes a context-budget decision

Every tool definition consumes attention and tokens. Exposing hundreds of low-quality tools can make selection harder and increase latency. Capability discovery, grouping, filtering, and deferred loading become part of interface design.

The question is no longer only, “Which APIs exist?” It is also, “Which capabilities should be visible in this context, for this task, to this caller?“

6. The execution model can vary

Current tool systems may defer discovery, filter tools per request, execute some tools on the provider side, issue parallel calls, or represent long-running work with a task handle. These are variations in discovery and execution, not a change to the authority boundary.

Someone still has to decide which operation is exposed, validate the request, enforce policy, handle side effects, and report what happened.

Where the Analogy Breaks

The RPC analogy is useful because it highlights a request, a contract, a remote operation, and a result. It becomes dangerous when it suggests that the caller has the same guarantees as ordinary code.

A valid request is not necessarily a correct request

Schema validation answers whether the arguments have an allowed shape. It does not answer whether the model selected the right order, interpreted the user’s intent correctly, or used a current value.

A description is not authorization

Tool metadata may explain an operation. It must not be the source of permission. The runtime needs an identity and policy decision that holds even when the model, prompt, or tool description is wrong.

A result is not automatically truth

The result may be stale, partial, malformed, or produced by an external system with its own uncertainty. The runtime should preserve source, timestamp, status, and error information where those details affect the next decision.

Retries are not harmless by default

Retrying a read may be reasonable. Retrying a refund, email, deployment, or record update may duplicate a side effect. Idempotency keys, operation status, and explicit compensation behavior matter.

Tool output is untrusted input

A tool can return text, markup, documents, or records that contain instructions. If that output is placed directly into model context, it can influence later decisions. A tool result needs an authority boundary just as retrieved web content does.

The caller can be confidently wrong

The model may select the most plausible tool rather than the correct one. It may fill an optional argument with an assumption. It may continue after an error because the response looks like a recoverable problem. Confidence in the generated text is not evidence that the operation is correct.

Under the Hood

A production tool call is more than a function name and a JSON object.

The model proposes an operation, but the runtime remains responsible for the boundary around its effects.

1. Define the capability contract

A useful contract includes at least:

• a stable name;
• a human-readable description of what the capability does and does not do;
• input schema and required fields;
• output schema or a documented result shape;
• side-effect classification;
• authorization requirements;
• idempotency and retry behavior;
• error categories that a caller can act on;
• latency, cost, and rate-limit expectations;
• confirmation and approval semantics where relevant;
• a completion model for asynchronous or long-running work;
• source and freshness information where the result represents external state.

JSON Schema is useful for expressing data shape. It is not a complete business contract. “amount is a number” does not mean “this caller may refund this amount for this order.”

2. Present the right capabilities

The host presents or exposes a set of available tools. That set may be static, filtered by task, selected by user role, discovered from another service, or loaded only when a category becomes relevant.

The smaller and clearer the set, the easier it is to evaluate selection. A broad registry can be powerful, but it makes descriptions, naming, collisions, and context budgets architectural concerns.

3. Receive a proposal

The model returns one or more structured proposals such as:

{
  "name": "get_order_status",
  "arguments": {
    "order_id": "A-1842"
  }
}

The format is structured, but the decision that produced it is still probabilistic. The execution host should treat it as an untrusted request from an intelligent but fallible caller.

4. Validate before execution

Validation should happen in layers:

1. Is the tool registered, enabled, and visible in this context?
2. Which user, workload, or delegated identity is the call made under, and does it have access?
3. Does the proposal satisfy the input schema?
4. Does it satisfy domain validation and current business policy?
5. Does the operation fit the current budget, rate limit, and side-effect rules?
6. Does it require a human approval step?

Only after those checks should the execution host invoke the capability.

5. Return an observable result

The result that re-enters model context should distinguish success, failure, and uncertainty. The execution host should preserve enough metadata for a later answer to avoid claiming more than the capability established.

For a read operation, that may mean source system, record identifier, retrieval time, and freshness. For a write operation, it may mean accepted, completed, rejected, or pending, plus an operation identifier that can be checked later.

6. Decide whether the loop continues

The runtime or model can decide that the interaction is complete, but the execution host should enforce hard bounds. Maximum tool turns, wall-clock time, token budget, cost budget, and repeated-call detection prevent a model from converting a recoverable ambiguity into an unbounded loop.

Tool calling sits between model context and governed capability, with evaluation, approval, budgets, tracing, and security around the path.

A Concrete Example

Consider a support assistant handling an order question.

The user says, “My order A-1842 is late. Can you refund the shipping charge?”

The assistant may need two capabilities:

• get_order_status, a read operation that returns shipment events and the current delivery state;
• request_shipping_refund, a consequential operation with eligibility rules and a possible approval requirement.

A safe sequence could be:

1. The model selects get_order_status and proposes A-1842.
2. The runtime validates that the support identity may inspect the order.
3. The order system returns the latest status, shipment history, source, and timestamp.
4. The model interprets the result and may propose request_shipping_refund with the order identifier and reason.
5. The capability or policy service evaluates current eligibility; the model does not establish it.
6. The execution host checks the support identity, order ownership, eligibility, refund limit, idempotency key, and approval rule.
7. The capability returns a confirmed refund, a pending review, or a rejection with a reason.
8. The model explains the result without claiming a refund was completed if the operation is only pending.

The model improves the interaction because it can connect an open-ended request to relevant capabilities. The execution boundary remains necessary because the model should not be trusted to enforce authorization, policy, eligibility, or side-effect safety.

The same pattern applies outside customer support. An IT assistant may inspect an incident before proposing a restart. A procurement assistant may gather invoice evidence before proposing approval. A deployment assistant may inspect a change and request execution. The names change, but the boundary does not: model proposal, independent validation, governed execution, observable outcome.

What Changes Because of It?

Architecture

The tool boundary becomes a first-class integration boundary. Teams need to decide which operations are exposed, how they are named, how they are grouped, which context is required, and where validation lives.

The right unit is usually a governed capability, not a raw internal function. The capability can wrap existing APIs and workflows while adding identity, policy, idempotency, audit, and outcome semantics.

Engineering

Tool descriptions become model-facing interface artifacts that need review and versioning. Schemas need compatibility discipline. Tool results need stable status and error behavior. The loop needs budgets and tracing.

Testing must include more than whether an API returns the expected response. It should measure whether the model selects the right tool, supplies correct arguments, asks for missing information, handles errors, stops when it should, and avoids tools that are outside the task.

Product and UX

Users need to understand when the system is reading information, proposing an action, waiting for approval, or reporting a completed outcome. A conversational interface can hide the operation path, but hiding it does not make the path less consequential.

For high-impact actions, confirmation should name the action, target, scope, and expected effect. “Proceed?” is not enough if the user cannot see what will happen.

Business and operations

Operations teams need ownership for capability definitions, policy changes, tool availability, and incident review. A tool can be technically healthy while the business rule around it is stale.

Latency and cost also become product concerns. One natural-language request may cause several model turns and external calls. A successful answer may be too slow or expensive for the experience even when each component is working as designed.

Security and governance

The tool surface is an authorization surface. Every capability should have an owner, an access policy, an audit trail, and a clear side-effect classification.

Prompt injection and untrusted tool output make the boundary more complicated. The model may see instructions in a document, web page, email, or tool result. The execution host should not allow those instructions to change authorization or bypass approval.

Failure Modes

The most common failures occur between the model’s semantic proposal and the execution boundary that decides what may happen.

Wrong tool selection

Two tools may have overlapping names or descriptions. The model selects a plausible operation that has the wrong scope. Improve naming, descriptions, tool filtering, and selection evaluations rather than assuming the model will learn the distinction from repetition.

Ambiguous arguments

The user names a person, account, or order incompletely. The model fills in a likely value. Require disambiguation when the cost of being wrong is higher than the cost of asking.

Schema success, business failure

The arguments pass JSON Schema but violate current policy, state, ownership, or eligibility. Business validation must remain inside the capability boundary.

Privilege confusion

The model sees a tool definition and assumes it may use it. The application exposes more capabilities than the user’s identity should be able to invoke. Filter the tool set and enforce authorization again at execution time.

Duplicate side effects

A timeout makes the caller uncertain whether an operation completed. The model retries. Idempotency keys and status lookup are safer than blind repetition.

Prompt injection through tool output

The tool returns content that tells the model to ignore its instructions or take another action. Treat tool output as data, preserve provenance, and enforce tool authorization outside the model’s context.

Unbounded tool loops

The model keeps searching, retrying, or calling a chain of tools because no stopping condition is explicit. Enforce turn, time, cost, and repetition budgets.

Stale or partial results

The model answers from a result without accounting for its timestamp, completeness, or source. Return status and provenance in a form the model can use, then test whether it communicates uncertainty correctly.

Contract drift

The implementation, schema, description, and examples evolve at different speeds. The model continues to make decisions from an outdated contract. Version the contract and include tool definitions in the same evaluation and release process as code.

KNOW / UNDERSTAND / BUILD

KNOW

Know that tool calling is the mechanism that lets a model propose a structured capability invocation and receive the result as context. Know that the model does not directly execute the operation and that an execution host must own the authority boundary.

UNDERSTAND

Understand the difference between semantic selection, schema validation, authorization, business policy, execution, and observation. Understand why tool descriptions influence behavior, why results can create another model turn, and why retries and side effects need explicit design.

BUILD

Build a small tool loop with one read operation and one write operation. Add structured schemas, authorization, a simulated ambiguous request, a tool error, a duplicate-call test, a budget, and a trace. Then inspect the calls rather than only the final answer.

Build It Once

Start with a narrow capability instead of exposing an internal service catalog.

For the support example, define get_order_status with a required order identifier and a result containing current status, events, source, and retrieval time. Define request_shipping_refund with an explicit order identifier, reason, and request identifier or idempotency key. If the caller supplies an amount, validate it against policy; otherwise calculate it inside the capability. Mark the second operation as consequential and route it through policy and approval.

The runtime should log:

• the user and application identity;
• the model and tool-contract version;
• the proposal received from the model;
• validation and authorization decisions;
• approval state;
• capability request and operation identifier;
• result status, source, and latency;
• the final answer shown to the user, subject to redaction and retention policy.

Then evaluate the loop with cases that are easy for a human to distinguish but easy for a model to confuse:

• an order number that belongs to another customer;
• a late order that is not refund-eligible;
• a request missing the order number;
• a timeout after a refund may have completed;
• an injected instruction in a shipment note;
• a user asking for an action they are not authorized to perform.

The smallest useful implementation teaches more than a large demo with twenty loosely defined tools. It exposes where the model is helpful and where the runtime must remain authoritative.

Will This Term Survive?

Tool calling is now a widely used label for a stable interaction pattern: a model proposes a structured operation, a system executes it, and the result returns to the model or user.

Function calling remains a common provider-specific name for a similar mechanism. Tool use is broader and often includes provider-hosted operations. Capability invocation is more general, but less familiar.

The exact label may change as platforms and protocols refine discovery, schemas, authorization, long-running tasks, and result handling. The underlying pattern is durable because model-mediated software needs a controlled path from interpretation to external effect.

The durable question is not which label wins. It is whether a system can expose useful capabilities without surrendering authority to a probabilistic caller.

Where It Fits in the Map

The map above shows tool calling at the capabilities edge of the series.

• Models and inference produce the proposal.
• Context engineering determines what the model can use to interpret the goal.
• Structured outputs and schemas constrain the proposed shape.
• Authorization and policy decide whether the operation may proceed.
• Agents and orchestration decide whether to call once, call again, or stop.
• Observability and evals reveal whether selection and execution behave well.
• MCP and other protocols can standardize parts of how capabilities are described, discovered, and invoked; they do not replace domain policy or authorization.
• Memory and state preserve the information needed across turns and operations.

The pattern is not a replacement for APIs, workflows, or application boundaries. It is a new participant at the boundary between an open-ended goal and a governed operation.

Sources

• Function calling, OpenAI API: a primary reference for defining and invoking model-proposed function calls.
• How tool use works, Anthropic Claude Platform Docs: a current reference for the tool-use request and result cycle.
• Tools, Model Context Protocol specification: a current reference for describing, discovering, and invoking tools through a protocol.
• Authorization, Model Context Protocol specification: a current reference for authorization semantics around protocol-hosted tools.
• Understanding JSON Schema: a primary reference for the schema constraints used to shape tool arguments.
• AIP-155: Request identification: a reference for idempotency keys and safe request retries.