Function Calling: The API Surface of Tool Use

Function calling is the typed proposal layer through which language models request actions outside their text-generation context. It exists across OpenAI, Anthropic, and Google APIs as similar-but-not-portable mechanisms that constrain model output to a JSON Schema and route it to application code; the production reliability of any system built on it depends less on the schema-validation channel and more on the harness that decides which proposed calls get executed.

Coverage note: verified through May 2026; benchmark numbers cite the Berkeley Function Calling Leaderboard V4 snapshot last updated April 12, 2026.

What function calling is

The phrase "function calling" describes a narrow technical mechanism and a much broader system pattern, and the distinction matters from the first paragraph. The narrow mechanism: a developer declares one or more callable functions to a language model, including each function's name, a natural-language description, and a JSON Schema for its parameters; the model, when generating a response, may emit a structured object naming one of those functions and a set of arguments. The broader pattern: that emitted object is a proposal — the model has not actually called anything. The application receives the structured object, validates it, decides whether to honor it, executes the underlying function in whatever environment it controls, and feeds the result back into the model so the conversation can continue.

Both OpenAI's and Anthropic's documentation are explicit about the loop. OpenAI's current guide describes the cycle as: declare tools, receive tool calls in the response, execute them in application code, send results back via a tool message, then receive a final response or further tool calls (OpenAI Function Calling). Anthropic frames it identically through tool_use and tool_result content blocks: the assistant produces a tool_use block with a name and an input object, the application executes it, and a subsequent user-role message returns a tool_result block referencing the same tool_use_id (Anthropic How Tool Use Works). Google's Gemini API uses functionDeclarations on the request side and emits a functionCall part containing a name, an arguments object, and an id; the application returns a functionResponse part for that id on the next turn (Gemini Function Calling).

In every case, the model does not "call" a function in the conventional software-engineering sense. It emits a candidate action that the host system may execute. This is not pedantry. Once an action can read private data, mutate state, spend money, send messages, browse untrusted content, or chain into other tools, the practical API boundary is no longer the JSON object the model produced — it is the policy-enforced execution boundary the host owns. A wiki article that teaches the JSON object as the essence will systematically understate the engineering problem.

The clean way to state the topic, then, is: function calling narrows the interface between a probabilistic generator and deterministic software from free text to structured intent. It does not make the intent correct, authorized, complete, or safe. The remaining sections trace where that distinction shows up.

The minimal protocol

A complete tool-use round trip in any of the major provider APIs has the same five-stage shape, with provider-specific names for each stage:

1. Declaration. The application sends a list of available tools with the request. Each tool has a name, description, and parameter schema. OpenAI calls these tools with type: "function" entries; Anthropic uses tools with name, description, and input_schema; Gemini uses tools.functionDeclarations with name, description, and parameters.
2. Selection. The model decides whether to respond with text, with a tool call, or with both. Each provider exposes a control over this choice (tool_choice in OpenAI and Anthropic; toolConfig.functionCallingConfig.mode in Gemini, with values such as AUTO, ANY, NONE, and the more recent VALIDATED).
3. Emission. The model emits a structured call: a function name and an arguments object. OpenAI returns these as tool_calls entries inside the assistant message, each with a call_id; Anthropic returns them as tool_use content blocks inside the assistant message, each with a tool_use_id; Gemini returns them as functionCall parts with an id.
4. Execution. The application receives the structured call, validates and authorizes it, executes the underlying code or API request, and produces a result.
5. Continuation. The application returns the result to the model on the next turn, identified by the same id, so the model can incorporate it into further reasoning. OpenAI uses a tool role message; Anthropic uses a user-role message containing a tool_result content block; Gemini uses a functionResponse part.

The full conversation is structured as an ordered transcript of these turns: an initial user message, an assistant message that may contain text and tool calls, one or more tool-result entries, then another assistant message, and so on, until the assistant produces a final answer with no further tool calls. In all three providers, the model's view of the world is limited to what is in this transcript; results not returned via a tool_result (or equivalent) are invisible to it.

This protocol shape is portable in spirit. The detailed semantics — schema dialect, strict-mode behavior, parallel-call support, when the SDK runs the loop for you, and what happens on malformed input — are not. Engineers migrating between providers should expect to rewrite both the schema layer and the loop driver, not just rename fields.

Provider surfaces

The three major surfaces (OpenAI, Anthropic, Google Gemini) are similar enough that the abstraction "function calling" makes sense, and different enough that a side-by-side reference matters.

Aspect	OpenAI Responses API	Anthropic Messages API	Google Gemini API
Tool list field	tools[] with type: "function"	tools[] with name, input_schema	tools.functionDeclarations[]
Schema dialect	JSON Schema (subset; see strict-mode constraints)	JSON Schema (subset; per-tool input_schema)	OpenAPI 3.0 schema subset
Model output for a call	tool_calls[] with call_id, function.name, function.arguments (JSON string)	Assistant content blocks of type tool_use with id, name, input (object)	Candidate parts of type functionCall with id, name, args (object)
Result return	tool role message with matching tool_call_id	User role message with tool_result block, matching tool_use_id	User role message with functionResponse part for the matching id
Forced selection	tool_choice: "auto" \| "none" \| "required" \| {type: "function", name}	tool_choice: {type: "auto" \| "any" \| "tool" \| "none", name?}	toolConfig.functionCallingConfig.mode: "AUTO" \| "ANY" \| "NONE" \| "VALIDATED"
Strict / constrained mode	strict: true per function; requires additionalProperties: false, all fields in required	strict: true per tool; uses grammar-constrained sampling for supported schemas	mode: "VALIDATED" (and ANY) constrains output to declared schema
Parallel calls	Supported; parallel_tool_calls: false to force at most one	Supported; multiple tool_use blocks per assistant message	Supported via multiple functionCall parts
Built-in / hosted tools	Web search, file search, code interpreter, image generation, computer use, MCP connectors as first-class type values in tools[]	Built-in web_search, code_execution, text_editor, bash, computer tools	Built-in Google Search, code execution, URL context
Native loop driver	Agents SDK; Responses API also supports stateful conversation IDs	Tool Runner; Claude Agent SDK	google-genai SDK supports automatic function calling for Python callables

Several of these distinctions deserve elaboration because they shape downstream code.

Schema dialect. OpenAI's strict mode supports a constrained subset of JSON Schema (no unsupported keywords, additionalProperties: false required, every property listed in required, no oneOf at the top level in some versions, and provider-side normalization that may rewrite a non-strict schema if strict: true is set or implied) (OpenAI Function Calling). Anthropic's strict tool use is grammar-constrained and accepts only schemas it can compile into a grammar; some keywords (e.g., pattern, complex anyOf, conditional schemas) are unsupported and will be rejected at request time (Anthropic Strict Tool Use). Gemini's parameters field accepts an OpenAPI-3.0-flavored subset; certain JSON Schema keywords have no equivalent and will be silently ignored or rejected. Cross-provider schemas almost never copy without modification.

Output shape. OpenAI returns function arguments as a JSON-encoded string, which the application must parse before use; Anthropic and Gemini both return arguments as parsed JSON objects. This is a frequent source of off-by-one bugs in cross-provider code. The string-based shape exists for legacy reasons (the original Chat Completions function-calling API streamed arguments token-by-token as text) and is preserved for compatibility.

Tool selection control. OpenAI's tool_choice: "required" forces the model to call some tool but not which one; Anthropic's tool_choice: {type: "tool", name: "X"} forces a specific tool; Gemini's mode: "ANY" forces a function call but allows the model to pick which function. These knobs are subtly different and benchmark results depend on which one is used.

Parallel calls. All three providers can return multiple tool calls in a single assistant turn — OpenAI as multiple entries in tool_calls[], Anthropic as multiple tool_use content blocks, Gemini as multiple functionCall parts. OpenAI documents that some fine-tuned model integrations or strict-mode combinations may force at most one call per turn, and parallel_tool_calls: false is the explicit knob to enforce that (OpenAI Function Calling). Application code that assumes exactly one call per turn will silently break when a model decides to issue two.

Built-in tools. All three providers have moved aggressively to first-class hosted tools, which the model invokes through the same protocol as developer-declared tools but whose execution happens inside the provider rather than in application code. From the model's perspective there is no distinction; from the developer's perspective there is a substantial one — built-in tools are subject to the provider's safety policies, billing model, and observability, not the application's.

Native loop driver. OpenAI's Agents SDK, Anthropic's Tool Runner and Claude Agent SDK, and Google's google-genai automatic function calling each let an application register Python (or TypeScript) callables and have the SDK run the request/execute/return loop on the developer's behalf. This is convenient and dangerous for the same reason: the loop's retries, error handling, state, and termination criteria become the SDK's defaults rather than the application's choice. For a one-off script this is fine; for a production system it should be treated as a starting point to be replaced with an explicit harness.

The honest summary is that the three providers expose the same conceptual primitive with enough operational divergence that "use function calling" is not a portable design decision. It is three design decisions sharing a label.

Strict mode and schema enforcement

Strict mode is the most discussed reliability feature in the function-calling literature, and it is the one most consistently overstated. What it does is real and useful; what it does not do is the larger half of the problem.

What strict mode actually does. When a tool is declared with strict semantics, the provider's serving infrastructure constrains the model's token-level output so that the resulting argument object is guaranteed to conform to the declared JSON Schema. OpenAI's docs describe this as adherence to the schema, requiring additionalProperties: false and that all top-level fields appear in required (OpenAI Function Calling). Anthropic's strict tool use uses grammar-constrained sampling to enforce schema conformance for supported tool schemas, with explicit guarantees about JSON validity, required fields, enum values, and type correctness (Anthropic Strict Tool Use). Gemini's VALIDATED mode constrains the model to emit a valid call against one of the declared function declarations.

The empirical effect is that the class of failures called "the model returned malformed JSON" or "the model hallucinated a field name" effectively disappears. Pre-strict-mode systems spent meaningful engineering effort on JSON-repair loops, ad hoc parsing, and try/except wrappers around json.loads(). Strict mode obviates most of that.

What strict mode does not do. It does not make the call correct. A strictly schema-conformant call can still:

• Reference the wrong customer, account, file, or resource id.
• Pick the wrong tool from a set of overlapping options.
• Pass a syntactically valid string that contains an injected instruction, an exfiltration URL, a SQL fragment, or a shell payload.
• Restate stale information from earlier in the conversation as if it were current.
• Make a perfectly typed request that violates a business rule the schema cannot express.

This is the syntactic-vs-semantic distinction, and it is empirically grounded. FuncBenchGen reports that even strong models routinely make schema-valid calls that propagate stale or wrong argument values across multi-step tasks; the paper finds that adding an explicit "restate the variable's current value" step before each call improved GPT-5's success rate from 62.5% to 81.3% in their setup (FuncBenchGen). IFEval-FC documents a related failure: BFCL-style argument-correctness checks do not exercise format instructions embedded in parameter descriptions, and even GPT-5 and Claude Opus 4.1 fail simple natural-language formatting constraints inside function-call arguments (IFEval-FC).

Strict mode's costs. The schema constraints are not free. There is some additional latency from schema processing and grammar compilation, particularly for the first request with a given schema (subsequent requests typically hit a cache). Schemas with deeply nested oneOf, anyOf, or conditional branches may be rejected outright by some providers and require flattening. Optional fields combined with additionalProperties: false and an all-required top level produce awkward declaration patterns where every "optional" field becomes a required nullable type. Strict mode also reduces the model's ability to gracefully degrade on out-of-schema cases — if the user asks for something the schema cannot express, the model is forced to either omit the request, refuse, or coerce it into an inappropriate shape.

The honest framing: strict mode trades flexibility for syntactic guarantees, and it solves a real but narrow class of failures. A tool surface that uses strict mode and declares no semantic invariants is in a worse position than a lenient-mode tool surface that validates against business rules — because strict mode produces calls that look correct enough to execute. A clean-shaped wrong call is more dangerous than a malformed wrong call, because the malformed one fails fast at the parser and the clean one fails late at the side effect.

Why validation remains hard

Once schema conformance is removed as the bottleneck, the remaining validation problem becomes the actual problem: deciding whether a clean, well-typed call is the right call to execute now. This decomposes into roughly four kinds of question.

Is this the right tool? Tool-selection failures are a recurring theme in the literature. Gorilla's original APIBench paper documented hallucinated APIs, wrong API selection, and inaccurate input arguments as the primary failure modes for general LLMs, and showed that retrieval-aware fine-tuning reduced but did not eliminate them (Gorilla / APIBench). On the Robustness of Agentic Function Calling reports that simply expanding a toolkit with related distractor tools degrades selection accuracy on otherwise-strong models (Rabinovich & Anaby-Tavor 2025). HammerBench finds parameter-name shifts and external-information dependencies produce sustained failures in multi-turn mobile-assistant scenarios (HammerBench). The pattern is robust: as the tool surface grows past a small handful, selection accuracy falls, and the fall is steeper for tools whose names and descriptions overlap.

Are the arguments semantically correct? Argument correctness is harder to evaluate than selection because it requires a ground truth about state. Did the model use the current customer id or one from three turns ago? Did it use today's date or the date in the example shown in the tool description? Did it correctly apply a transformation the user requested in natural language, or did it apply a different one that happens to also fit the schema? FuncBenchGen's stale-value finding belongs here, as does most of the multi-turn / state-dependent failure literature.

Is the call appropriate to make right now? Even a correct tool with correct arguments may be wrong to execute. The user may be exploring rather than deciding; a confirmation step may be required; a previous side effect may not yet be reflected; the call may be redundant with one already in flight. None of these are visible in the schema or in the JSON.

Is the call safe to execute? This is where injection and excessive agency live, and we treat it in its own section below.

The practical consequence is that schema validation is a necessary but small fraction of the validation budget. A useful tool harness layers:

• Parser validity (is this JSON? — usually free with strict mode).
• Schema validity (does it conform to the declared types? — strict mode plus a server-side validator).
• Business-rule validity (does the call satisfy invariants the schema cannot express? e.g., the date is in the past, the recipient is in the user's contacts, the amount is below the daily limit).
• Authorization (does the calling identity have permission to take this action against this resource?).
• Approval policy (does this action require explicit human confirmation before execution?).

Skipping any of these in production produces a different shape of incident. Skipping business-rule validation produces silent semantic errors. Skipping authorization produces privilege escalation. Skipping approval produces irreversible side effects on a probabilistic actor's say-so.

Reliability evidence and benchmark limits

The empirical literature on function-calling reliability has matured fast, and the primary lesson is that benchmark construction dominates what "reliability" appears to mean.

Berkeley Function Calling Leaderboard (BFCL). The canonical public benchmark, now at V4. Per the leaderboard snapshot last updated April 12, 2026, top overall scores remain below 80%: Claude Opus 4.5 (FC) at 77.47%, Claude Sonnet 4.5 (FC) at 73.24%, Gemini 3 Pro Preview (prompt mode) at 72.51%, and Gemini 3 Pro (FC) at 68.14% (BFCL V4 Leaderboard; BFCL Data CSV). Headline rankings hide a wide spread across categories: simple single-turn AST-evaluated tasks score in the high 80s to mid 90s for top models, while multi-turn and memory categories sit considerably lower. The Berkeley technical report frames function-invocation evaluation as fundamentally hard because deterministic correctness often requires execution; BFCL works around this with AST-style validation proxies, which are useful but not equivalent to executable end-state correctness (BFCL Technical Report). The leaderboard's scoring composition also changed in V4 to weight agentic tasks more heavily, while some descriptive language about an unweighted average lingered; the changelog is the authoritative source of truth (BFCL Changelog).

ToolSandbox. Argues that earlier function-calling benchmarks over-focused on stateless, single-turn, off-policy evaluation, missing state dependencies, canonicalization issues, insufficient-information scenarios, and on-policy conversational behavior (ToolSandbox). On those richer dimensions, frontier-model performance is meaningfully worse than on BFCL-style single-turn benchmarks.

HammerBench. Tests multi-turn, mobile-assistant-style scenarios with parameter-name shifts, argument substitution, and external-information dependencies; reports sustained failures across model families (HammerBench).

WildToolBench. Evaluates user-behavior-grounded multi-turn tool use and reports no evaluated model above 15% accuracy on its hardest setting (WildToolBench). The number should be read carefully — the benchmark is deliberately adversarial in its construction of realistic, messy user behavior — but it is a useful counterweight to the implication that high BFCL scores translate to production reliability.

WildAGTEval. Finds that realistic API complexity, especially the presence of irrelevant information in surrounding context, can substantially reduce strong-model performance (WildAGTEval).

ReliabilityBench. Reports that perturbations of input prompts cause sustained accuracy drops; success rate falls from 96.9% at no perturbation to 88.1% at perturbation intensity 0.2, with rate-limiting failures particularly damaging under fault injection (ReliabilityBench).

IFEval-FC. Argues argument-correctness benchmarks like BFCL do not test format instructions embedded inside parameter descriptions, and demonstrates that even GPT-5 and Claude Opus 4.1 fail simple format constraints expressed in natural language inside the schema (IFEval-FC).

FuncBenchGen. Multi-step tasks with stale-value propagation; the finding that explicit variable restatement raised GPT-5's success rate by ~19 percentage points is significant because it suggests a substantial fraction of failures are recoverable via cheap prompt-level interventions (FuncBenchGen).

Benchmark	What it tests	Approx. frontier model score	Headline limitation
BFCL V4	Single-turn and limited multi-turn function-calling, AST-validated	68–78% overall	AST proxies, leaderboard categories not equally weighted in headline
ToolSandbox	State-dependent, on-policy, conversational	Lower than BFCL on equivalent models	Smaller universe, harder reproducibility
HammerBench	Multi-turn mobile-assistant, parameter shifts	Sustained failures across families	Mobile-domain specific
WildToolBench	Adversarial multi-turn user behavior	<15% best	Deliberately worst-case
WildAGTEval	Realistic API complexity, distractor information	Substantial drops vs clean	Smaller scale
ReliabilityBench	Perturbations and fault injection	88.1% at intensity 0.2 (vs 96.9% baseline)	Synthetic perturbation distribution
IFEval-FC	Format-following inside parameter descriptions	GPT-5 / Claude Opus fail simple cases	Targeted at one failure mode
FuncBenchGen	Multi-step stale-value propagation	GPT-5: 62.5% → 81.3% with restatement	Single intervention measured

The right reading of this table is not "BFCL is the reliable benchmark and the others are pessimistic." It is that single-turn, clean-schema benchmarks measure a different skill from stateful, adversarial, permissioned, user-messy workflows. A wiki article, an engineering decision, or a vendor pitch that collapses these into a single reliability number is laundering benchmark validity into product reliability.

The other consequence is operational: any time a leaderboard score is cited in a product or research context, it should travel with the benchmark version, the date the snapshot was taken, the category of task evaluated (single-turn vs multi-turn vs agentic), the model snapshot tested (FC vs prompt mode), and ideally a link to the underlying CSV. BFCL is a live, versioned artifact; numbers from six months ago are not directly comparable to numbers from today.

Integration topologies

Function calling is the wire-level primitive. How a system arranges multiple calls into a unit of work is a separate design decision, with three common shapes and meaningful differences in failure modes.

One call per turn. The model emits at most one tool call per assistant message; the application executes it and returns the result; the model then issues the next call or a final response. This is the simplest pattern and has the cleanest failure semantics — every call has a clear before/after state, retries are localized, and human-in-the-loop confirmation is straightforward. It is also the slowest because every call costs a model round-trip. OpenAI's parallel_tool_calls: false and Anthropic's tool_choice: {type: "tool", name} (combined with surrounding control flow) are the explicit knobs to enforce this discipline.

Parallel independent calls. The model emits multiple tool calls in one assistant message; the application executes them concurrently and returns all results in the next user-role turn. This is supported by all three major providers and is genuinely useful when the calls are independent — fetching three rows from three databases, querying three search backends in parallel, fanning out independent sub-questions. The failure modes are also genuine: race conditions if the calls share state, duplicated side effects if a parallel batch is retried partway through, partial failures that leave the assistant reasoning over an incomplete worldview, and harder rollback semantics. Treating parallel tool calls as "just a latency optimization" understates the engineering cost; they should be reserved for genuinely independent reads, with side-effecting tools defaulted to serial execution and explicit idempotency keys.

Sequential / nested calls through the harness. The model issues a call, sees the result, and uses the result to choose the next call. The "nesting" is in the conversation, not in the model — the model does not literally execute a subroutine. This is the most powerful and most failure-prone pattern, because each step's correctness depends on the previous step's output, and the chain can drift. Stale-value bugs of the kind FuncBenchGen documents arise predominantly here (FuncBenchGen).

Long-running agent loops. A specialization of nested calls where the application runs a loop until some termination condition is met (the model emits no further tool calls, a maximum step count is reached, a budget is exhausted, or an external signal is received). All major providers ship SDKs that automate this loop — OpenAI's Agents SDK, Anthropic's Tool Runner and Claude Agent SDK, Google's automatic function calling — but the SDK defaults are typically generous (high step limits, automatic retries on transient errors, broad tool registration). For production, the loop driver should be owned by application code, with explicit budgets, observability, and termination conditions. See Agent Harness Architecture for the broader treatment.

A few patterns are common to all of these and worth stating explicitly:

• Idempotency keys. Any side-effecting tool should accept an idempotency key generated by the application, not the model. Using a model-generated key is a frequent source of duplicated effects, because the model may reuse the same key across legitimate distinct calls or generate a fresh key on a retry of an already-completed call.
• Tool-result truncation. Tools that can return large responses (web search, file reads, database queries) must have an application-side truncation policy. Returning a 200KB result into the conversation transcript will silently drive up cost and degrade reasoning quality on subsequent turns.
• Loop termination. Hard budgets matter: maximum step count, maximum total token spend, maximum wall-clock time. A model that fails to terminate is a bug, not a curiosity. Without budgets, a stuck loop will burn through the API account.
• State snapshots between turns. For conversations that involve mutating state (file edits, database writes), snapshotting the relevant state after each tool result lets the harness detect when the model is reasoning over a stale view.

These are not function-calling features; they are the surrounding control plane that makes function calling production-viable.

Security implications

The security analysis of function calling is the section where the framing most affects the conclusions. If function calling is treated as "the model calling functions," the natural threat model is "what if the model emits a malformed call?" — and strict mode answers most of that. If function calling is treated as "the typed proposal layer through which a probabilistic generator requests privileged actions," the threat model is much larger, and strict mode answers a small slice of it.

The literature has converged on a roughly stable taxonomy.

Tool-call argument injection. A malicious payload that fits the declared schema. If a tool accepts { "email_body": "..." }, an injected instruction inside email_body is schema-conformant. If a tool accepts { "url": "..." }, an exfiltration URL is schema-conformant. If a tool accepts { "message": "..." }, a data-leaking message is schema-conformant. Strict mode makes such calls easier to execute, not harder, because the call passes parser and schema validation cleanly. Defense lives in semantic validators (allowlists for hostnames, content scanners for outbound message bodies, policy checks before sending), not in the schema layer.

Prompt injection through tool results. Tool outputs become part of the conversation transcript and influence the model's next decision. A web-search tool that returns an attacker-controlled page can carry instructions that the model interprets as part of its task. OpenAI's safety guidance describes this directly: prompt injections are untrusted data attempting to override behavior, including via downstream tool calls and exfiltration (OpenAI Agent Safety Guide). The mitigation is a combination of provenance tagging (mark which conversation turns came from untrusted sources), output sanitization, restricted tool surfaces in untrusted-content branches of the conversation, and explicit instructions to the model not to follow instructions from tool outputs.

Excessive agency. The OWASP LLM Top 10 category (LLM06) is direct: LLM systems often get authority to call functions or plugins, and recommends minimizing the tool set, minimizing per-tool permissions, requiring approval for high-impact actions, and enforcing authorization downstream rather than relying on the LLM (OWASP LLM06: Excessive Agency; OWASP LLM Top 10). The recommendation is structural: the right way to prevent the model from doing something is to deny it the capability, not to instruct it not to.

Function-call jailbreaks. "The Dark Side of Function Calling" reports a jailbreak-via-function-call attack with average attack success above 90% across six models including GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro (Wu et al. 2024). The mechanism exploits the often weaker safety alignment of the function-arguments channel relative to the natural-language channel — the same model that refuses a request in chat may comply with it when the request is recast as a tool call.

Tool selection injection. ToolHijacker demonstrates prompt-injection attacks targeting the tool-selection step rather than the argument step. By embedding malicious content in tool descriptions or in surrounding documents the agent reads, an attacker can bias the model toward attacker-chosen tools (ToolHijacker). The defense is treating tool-description content with the same skepticism as user content — descriptions should be authored by the application owner, not concatenated from user-controlled sources.

Adaptive adversarial pressure. LLMail-Inject reports a public competition with 208,095 attack submissions from 839 participants targeting email-assistant agents to trigger unauthorized tool calls, and finds that models and defenses degrade meaningfully under adaptive attack (LLMail-Inject). The base-rate evidence here is sparse and the worst-case evidence is alarming.

Confused deputy. A class of failures where the model executes a call on behalf of an identity it should not be acting for, or against a resource it should not be touching, because the authorization check ran against the wrong principal. This is not specific to LLMs but is amplified by them — the model has no native concept of "whose request is this?" and will happily route a privileged operation through a confused authorization layer.

The corresponding defense pattern is also fairly well established in the literature. CaMeL ("Defeating Prompt Injections by Design") is notable because its defense is not a better tool-call validator; it separates control flow from data flow and enforces capabilities at the tool-call boundary, with the explicit position that schema-level mitigations are insufficient (CaMeL). The pattern generalizes: the right architectural answer to "schema-valid attacks succeed against schema-level defenses" is to push enforcement to a layer that can reason about identity, capability, and provenance, not to make the schema stricter.

A practical security checklist for any production tool-using system:

Control	Why it matters
Least-privilege tool design	The smallest tool surface the use case allows; broad action-enum tools become privilege ladders
Authorization downstream of the model	The tool executor checks whether the caller may perform this action on this resource; never trust the model to enforce its own permissions
Idempotency keys application-generated	Prevents duplicated side effects on retry, including injection-driven retries
Allowlists for high-risk argument types (URLs, hostnames, recipients)	Schema cannot express "is this URL safe to fetch" or "is this email address allowed"
Content scanning on outbound side effects	Catches exfiltration that fits the schema
Approval boundary for irreversible / high-impact actions	The class of action where the cost of a wrong call is greater than the cost of asking
Provenance tagging on inputs	Mark which transcript turns came from untrusted sources; restrict tool availability in those branches
Audit logging of every tool-call proposal and execution	Necessary for incident response; the tool-call channel is where text generation becomes system authority

Most production incidents in tool-using systems trace to one of: a tool surface broader than necessary, an authorization check that ran against the wrong principal, a side effect that was not idempotent, or an untrusted input branch that retained access to a privileged tool. None of these are problems strict mode can solve.

Relationship to agent harnesses

A consistent theme so far is that function calling is the model-facing primitive and agent harnesses are where the actual production system lives. The relationship deserves a section on its own because the line between "use the SDK's loop driver" and "build your own harness" is one of the most consequential design decisions in the area.

What native tool-use loops do. Each major provider ships an SDK with a loop driver that automates the request-execute-return cycle. OpenAI's Agents SDK lets a developer register Python callables and let the SDK manage tool dispatch, conversation state, and termination. Anthropic's Tool Runner and Claude Agent SDK provide the same shape with Anthropic-specific defaults. Google's google-genai library supports automatic function calling for Python callables. These are genuinely useful for prototyping, examples, and small applications.

What native loops hide. The defaults of any SDK loop driver are a set of judgments: how many steps before terminating, how to handle a transient error, how to surface a tool exception to the model, what to do when a tool returns a result larger than the context window can absorb, whether to retry on rate-limit errors, how to handle parallel calls in the presence of failures, how to charge the conversation for tool failures. Each of these is a defensible default and a wrong one for some applications.

The most common production failures from over-reliance on SDK loop drivers:

• Runaway costs. A model that gets stuck in a "let me try this slightly different call" pattern can burn through thousands of tool calls before the SDK's default step limit kicks in.
• Silent error swallowing. SDKs often catch exceptions in tool execution and feed an error string back to the model, which is the right default for resilience and the wrong default for observability — the application never sees that the database is down because the model "handled" the error by retrying.
• Context-window exhaustion. Tool results that the SDK forwards verbatim into the transcript can drive the conversation into a state where every additional call is more expensive and lower-quality, with no application-visible signal until the context limit is hit.
• Parallel-call failure modes. Default loop drivers may execute parallel calls without partial-failure semantics, leaving the model to reason over an inconsistent worldview when one of three parallel calls failed.

The framing that follows from this is: native loops are a starting point. A production agent harness owns:

• Termination policy. Hard step limit, hard token budget, hard wall-clock budget, deterministic exit conditions.
• Error surfacing. Tool exceptions reach application monitoring, not just the model. Some classes of error (auth failures, integrity violations) should terminate the loop rather than be retried.
• State management. The harness, not the model, owns the canonical state of any mutated resource between turns.
• Tool registry. Tools are registered by the application with explicit per-tool authorization and approval policies, not picked up implicitly from a Python decorator.
• Observability. Every tool-call proposal is logged with the call id, the proposed arguments, the validation result, the authorization decision, the execution result, and the latency.
• Compaction policy. When the transcript grows past a threshold, the harness decides what to summarize, what to drop, and what to keep verbatim.

See Agent Harness Architecture for the systems-level treatment, and Multi-Model Agent Orchestration Patterns for orchestration patterns where multiple models share a tool surface.

When function calling is the wrong tool

Tool use is not always the right framing. Several adjacent options exist and are sometimes preferable.

Structured outputs without execution. If the model needs to produce structured data but no action follows, use structured-output / JSON-mode features rather than declaring a no-op tool. OpenAI's response_format: {type: "json_schema", ...}, Anthropic's structured response patterns, and Gemini's responseSchema constraint all produce schema-conformant data without the conversational overhead of a tool round trip. This is faster, cheaper, and avoids the false implication that the model executed something.

Retrieval as context, not as a tool. A common anti-pattern is exposing a search_documents tool when the application already knows it needs document context. Pre-fetching the relevant documents and putting them in the prompt is faster, cheaper, and more reliable than asking the model to issue a tool call to retrieve them. The tool-based pattern wins when the decision to retrieve is the model's to make; otherwise, it is overhead.

Direct API integration, no model in the loop. If the user request is precise enough that deterministic code can construct the API call directly, the model's role should be intent-classification, not call generation. A natural-language interface with a small set of clearly defined intents may be better served by classifying the intent and invoking the API with code-built arguments than by handing the model a tool definition.

Workflow engines for fixed sequences. If the sequence of tool calls is fully determined by the application, putting the model in the middle of every step is overhead. Define the workflow in code, invoke the model only at decision points where its judgment is actually needed.

The general principle: tool calling is appropriate when the choice of action genuinely depends on the model's reasoning over the current conversation state. When the choice is determined or can be determined cheaply by code, use code.

Practical design rules

Distilled from the preceding sections, applicable across providers:

1. Treat the schema as syntactic, not semantic. Strict mode is a cheap and useful tool for one class of failure. It does not replace business-rule validation, authorization, or approval flows.
2. Keep the tool surface small. Selection accuracy degrades as tool count and overlap grow. Consolidate where possible without creating broad-action enum tools that smuggle privilege escalation through a single function name.
3. Generate idempotency keys application-side. Never trust the model to produce a key whose uniqueness across retries actually matters.
4. Default side-effecting tools to serial execution. Reserve parallel calls for read-only operations or operations with strong independence guarantees.
5. Truncate tool results before they enter the transcript. Set a per-tool maximum, document it in the tool description, and have the model request more if needed.
6. Own the loop driver in production. SDK loop drivers are good defaults; they are not policy. Wrap them or replace them once an application moves past prototype.
7. Set hard budgets. Maximum step count, maximum token spend, maximum wall-clock time. A loop without budgets is a bug waiting to invoice you.
8. Authorize downstream of the model. Permission checks happen in the tool executor against the calling identity, never in the prompt to the model.
9. Allowlist high-risk argument types. URLs, hostnames, recipient addresses, file paths, command strings — any argument whose space includes dangerous values needs an allowlist or content scanner.
10. Tag input provenance. Mark which transcript turns came from untrusted sources (web search results, user-uploaded documents, third-party tool outputs); restrict tool availability in branches where untrusted inputs are present.
11. Log every proposal and every execution. Tool-call telemetry is the security perimeter; without it, post-incident analysis is guesswork.
12. Pin and re-verify benchmark claims. When citing function-calling reliability, pin the benchmark version, the date of the snapshot, the model snapshot tested, and the category of task. Numbers age fast.

These are not exotic — they are the operating discipline that distinguishes a function-calling demo from a function-calling system in production.

What remains genuinely hard

Several problems in this area are not solved and are not well-mitigated by the current generation of provider features or harness patterns.

Multi-turn state tracking under ambiguity. When a conversation involves multiple resources, multiple intents, and references that resolve only in context (the user says "send it to her" — to whom?), models routinely lose track of which referent applies. FuncBenchGen's stale-value finding is one symptom; the broader pattern is that models are weaker at maintaining coherent state across many turns than they are at any single-turn task (FuncBenchGen).

Tool overload. As tool counts climb past roughly two dozen, selection accuracy degrades on every model family (Rabinovich & Anaby-Tavor 2025). The current best practice is to limit tool surfaces aggressively or to use a two-stage selection (a fast model picks a candidate set, the main model picks among them), but neither solution is fully satisfactory.

Adaptive adversarial pressure. The LLMail-Inject competition's findings suggest that defense techniques designed against fixed attack patterns degrade quickly when the attack pattern adapts (LLMail-Inject). The state of the art for adversarial robustness in tool-using agents is meaningfully behind the state of the art in capability.

Contamination and benchmark gaming. Several function-calling benchmarks now appear in training data, which complicates inference from leaderboard scores to held-out performance. The Berkeley team's response has been to maintain BFCL as a live, versioned artifact with regular refreshes, but the structural problem persists.

Cost vs reliability tradeoff at scale. Most reliability improvements in this area (more agent steps, more validation passes, more redundant queries) cost tokens. There is no public benchmark that systematically measures the Pareto frontier between cost and reliability across providers; engineering decisions in this space are made on intuition and per-application measurement.

These are not reasons to avoid function calling. They are reasons to deploy it with calibrated expectations about where it works and where it requires substantial surrounding engineering.

A note on terminology

The terms "function calling" and "tool use" overlap and are sometimes used interchangeably. The historical pattern: OpenAI introduced "function calling" in the Chat Completions API in 2023; Anthropic introduced "tool use" with tool_use content blocks; Google adopted "function calling" terminology aligned with OpenAI's. The Responses API era at OpenAI uses "tools" as the umbrella term, with function tools as one variety alongside built-in tools (web search, file search, code interpreter, MCP connectors).

This article uses "function calling" for the structured-call mechanism and "tool use" for the broader pattern that includes built-in tools, hosted tools, and the surrounding execution boundary. When the literature uses one term where the other would fit, the substantive content is almost always portable.

A related terminological note: "agent" is used in this article narrowly to mean a system that runs a loop of (model call → tool execution → model call → ...) until termination. Provider SDKs use "agent" more broadly to mean any application that uses their API; that broader usage is fine in marketing but unhelpful in technical writing because it conflates the loop-driving system with any LLM application.

Summary

Function calling is the typed proposal layer through which language models request actions. It is mature enough to be a standard integration primitive, with broadly equivalent shapes across OpenAI, Anthropic, and Google, and with strict-mode features that have largely solved the malformed-JSON class of failures. It is not mature enough to be a reliability boundary or a security boundary. Frontier models still fail in exactly the places production systems care about most: multi-turn state, tool overload, perturbations, ambiguous user intent, and schema-valid but semantically wrong or unsafe calls.

The right way to design a system that uses function calling is to treat the model's emitted call as a candidate action and to invest in the harness that decides which candidates become real. That investment — least-privilege tool design, downstream authorization, application-side idempotency, semantic validators, hard budgets, observability, and an approval boundary for irreversible actions — is where the actual reliability and safety properties of a tool-using LLM system live. The schema is a useful constraint on one channel of failure. The harness is the API surface of tool use.

Companion entries

Core mechanism: - JSON Schema for LLM Outputs - Structured Outputs and Constrained Generation - Provider API Comparison: OpenAI Anthropic Google - Strict Mode and Grammar-Constrained Decoding

Practice: - Agent Harness Architecture - Multi-Model Agent Orchestration Patterns - Idempotency Keys for LLM Tool Calls - Tool Result Truncation and Compaction - Parallel vs Serial Tool Execution - Approval Boundaries for High-Impact Actions - Native Tool-Use Loops vs Custom Harnesses - Context Window Economics

Reliability and evaluation: - Berkeley Function Calling Leaderboard - Multi-Turn Function Calling Benchmarks - Benchmark Validity and Production Reliability - Tool Selection Under Toolkit Expansion

Security: - Prompt Injection Through Tool Results - Tool-Call Argument Injection - Excessive Agency in LLM Systems - Confused Deputy Problems in Agents - OWASP LLM Top 10 - CaMeL Capability-Based Defense - Function-Call Jailbreaks

Counterarguments: - When Structured Outputs Beat Tool Calls - Direct API Integration vs Model-in-the-Loop - The Limits of Schema as Validation