Toolformer is a 2023 self-supervised method for teaching a language model to decide when to call external tools, what arguments to pass, and how to incorporate tool results into future token prediction. Its central contribution is a utility filter: generate candidate API-call demonstrations from a few examples, execute the calls, and keep only those whose returned result improves the model’s next-token loss; empirically, this improved GPT-J-scale zero-shot performance on math, factual lookup, temporal, and multilingual QA while exposing sharp limits for chained, interactive, and out-of-distribution tool use. arXiv+2arXiv+2
Coverage note: verified through May 18, 2026.
Why Toolformer mattered
Toolformer: Language Models Can Teach Themselves to Use Tools sits at an important hinge point in the history of Tool Use in LLMs. Before the paper, tool use in language models was mostly treated as a system-design problem: build a retriever, browser, calculator, Python interpreter, or API router around a model, then prompt or supervise the model to call it. Toolformer reframed the problem as a language-modeling problem: can a pretrained LM create its own tool-use supervision from raw text and then internalize a policy for when and how to call tools?
The answer, in Schick et al.’s experiments, was “partly yes.” The model did not learn a general agent loop, and it did not learn arbitrary APIs. But it did learn a practical behavior that later became central to the tool-use era: emitting structured tool calls inside generation, waiting for an external system to execute them, and conditioning subsequent tokens on the returned result. The paper’s canonical examples include fact lookup through a QA API, arithmetic through a calculator, translation through an MT system, and search through a Wikipedia API; the model autonomously inserts these calls where they help complete text. arXiv
Historically, Toolformer belongs beside ReAct as one of the foundational papers for modern LLM tool use. ReAct showed that an LLM could interleave reasoning traces and actions through prompting; Toolformer showed that a model could be fine-tuned on self-generated, utility-filtered tool-call traces and later decide when to use tools without task-specific demonstrations at inference. arXiv
Core thesis
Toolformer’s thesis is simple and technically sharp:
A language model can learn tool use from its own predictions if candidate tool calls are filtered by whether the tool result makes future tokens easier to predict.
This is different from ordinary supervised learning. The model is not trained on a large human-labeled corpus of “correct” API calls. It starts with only a handful of demonstrations for each API, samples candidate calls inside ordinary text, executes those calls, and keeps the calls that reduce weighted cross-entropy on the following tokens. The retained calls become a new language-modeling dataset. arXiv
The key idea is not “use tools.” Earlier systems already used search, browsers, calculators, retrieval modules, and program interpreters. The key idea is filter-by-utility self-supervision: the label is not whether a human thinks a tool call is sensible, but whether the model’s own predictive loss improves when the tool result is inserted.
The system in one pass
Toolformer begins with a pretrained causal language model. In the paper, the main model is GPT-J with 6.7B parameters, fine-tuned on a subset of CCNet augmented with API calls. The tools are constrained to text-in/text-out APIs, so a tool call can be linearized as ordinary tokens and inserted into the model’s context. arXiv
The five tool families used in the paper were:
| Tool | Purpose | Implementation detail in the paper |
|---|---|---|
| Question answering | Factoid lookup | Atlas, a retrieval-augmented LM fine-tuned on Natural Questions |
| Calculator | Arithmetic | Four basic arithmetic operations, rounded to two decimals |
| Wikipedia search | Broader factual retrieval | BM25 over a KILT Wikipedia dump |
| Machine translation | Translate non-English phrases into English | 600M-parameter NLLB with fastText language detection |
| Calendar | Temporal grounding | Returns the current date |
These were intentionally modest tools. Toolformer was not a general API-use benchmark; it was a demonstration that a language model could learn a heterogeneous set of tool-call formats through a unified text interface. arXiv
During inference, generation proceeds normally until the model emits the special “arrow” token indicating that it expects an API result. The system then interrupts decoding, executes the requested API call, inserts the result and closing API token, and resumes generation. This is the same high-level pattern later standardized in native tool-calling APIs: the model emits a structured call, an external runtime executes it, and the result is fed back into the model. arXiv
The algorithmic contribution: utility-filtered self-supervision
The central algorithm can be summarized as four stages.
| Stage | What happens | Why it matters |
|---|---|---|
| 1. Sample candidate tool calls | Prompt the LM with a few examples of a tool-use format, then let it annotate raw text with possible API calls. | This creates candidate demonstrations without human labeling at scale. |
| 2. Execute the calls | Run each candidate call through the corresponding external tool. | The system obtains actual tool outputs rather than merely synthetic placeholders. |
| 3. Filter by predictive utility | Keep only calls where the tool result lowers weighted next-token loss over following tokens. | This converts raw model guesses into self-supervised training data. |
| 4. Fine-tune on augmented text | Insert retained calls into the original corpus and continue language-model training. | The model learns when to emit calls, how to format arguments, and how to use outputs. |
The paper represents an API call as c = (a_c, i_c), where a_c is the API name and i_c is the input. A call without a result is linearized as:
<API> a_c(i_c) </API>
A call with a result is linearized as:
<API> a_c(i_c) → r </API>
In practice, the authors used existing token sequences such as [ and ] rather than modifying the vocabulary, but the conceptual interface is a special-token API span. arXiv
The utility filter compares two losses. Let L_i⁺ be the weighted loss on future tokens if the model is given the API call and its result. Let L_i⁻ be the better of two baselines: no API call at all, or the API call without its result. The candidate is kept only if:
L_i⁻ − L_i⁺ ≥ τ_f
The baseline that includes the call without the result is important. It prevents the model from keeping calls merely because the text of the call itself hints at the continuation. A retained call must make the future easier to predict because of the returned result, not just because the query is informative. arXiv
This is the heart of Self-Supervised Learning in Toolformer. The system does not need dense human annotations saying “call the calculator here” or “search Wikipedia here.” It uses the model’s own next-token loss as a proxy for whether the tool call was useful. That proxy is powerful, but it is also the source of several limitations discussed below.
What Toolformer actually learns
Toolformer learns at least four distinct behaviors.
First, it learns tool-call placement: where in a sequence an API call should occur. The training process samples possible positions and keeps those where the inserted result helps predict nearby future text. arXiv
Second, it learns tool selection: whether the current context is better served by a calculator, QA model, search engine, translation model, calendar, or no call. This is not a separate classifier; it is implicit in next-token generation. The model emits the token sequence corresponding to a tool call only when its learned distribution favors doing so.
Third, it learns argument construction: how to ask the question, write the arithmetic expression, phrase the search query, or choose the text to translate. This is learned from a mixture of few-shot prompting during data generation and fine-tuning on retained examples.
Fourth, it learns result incorporation: how to condition subsequent language on the returned tool output. The output is not merely appended as metadata; it becomes part of the causal context from which the model continues generating.
This combination is what made the paper more than a retrieval trick. Retrieval-augmented systems often decide outside the model when to retrieve. Toolformer tries to train the model itself to decide whether retrieval, calculation, translation, or temporal lookup is useful. That makes it a precursor to Native Tool Calling APIs and agentic tool loops, even though its own experiments are much narrower.
Empirical results
Factual completion: LAMA
Toolformer was evaluated on SQuAD, Google-RE, and T-REx subsets of LAMA, framed as left-to-right factual completion. Because LAMA statements were derived from Wikipedia, the authors disabled Wikipedia search for this evaluation and relied primarily on the question-answering tool. Toolformer’s GPT-J-based model outperformed GPT-J, GPT-J further trained on CCNet, Toolformer with tools disabled, OPT-66B, and GPT-3-175B on the reported LAMA subsets. arXiv
| Model | SQuAD | Google-RE | T-REx |
|---|---|---|---|
| GPT-J | 17.8 | 4.9 | 31.9 |
| GPT-J + CC | 19.2 | 5.6 | 33.2 |
| Toolformer, tools disabled | 22.1 | 6.3 | 34.9 |
| Toolformer | 33.8 | 11.5 | 53.5 |
| OPT 66B | 21.6 | 2.9 | 30.1 |
| GPT-3 175B | 26.8 | 7.0 | 39.8 |
The striking result is not just that Toolformer improved over GPT-J. It is that a 6.7B-parameter model, once given a self-supervised QA-call policy, beat much larger parameter-only baselines on these factual completion tasks. The paper reports that Toolformer used the QA tool for 98.1% of examples in this setting. arXiv
Math: ASDiv, SVAMP, MAWPS
The math results are the cleanest demonstration of the tool-use thesis. Arithmetic is exactly the kind of capability where a small external symbolic tool can outperform a large parametric model. On ASDiv, SVAMP, and MAWPS, Toolformer used the calculator for most examples and substantially outperformed GPT-J, OPT-66B, and GPT-3-175B. arXiv
| Model | ASDiv | SVAMP | MAWPS |
|---|---|---|---|
| GPT-J | 7.5 | 5.2 | 9.9 |
| GPT-J + CC | 9.6 | 5.0 | 9.3 |
| Toolformer, tools disabled | 14.8 | 6.3 | 15.0 |
| Toolformer | 40.4 | 29.4 | 44.0 |
| OPT 66B | 6.0 | 4.9 | 7.9 |
| GPT-3 175B | 14.0 | 10.0 | 19.8 |
The disabled-tools row matters. Toolformer improved somewhat even when API calls were disabled, presumably because fine-tuning exposed it to many examples containing calculations and results. But enabling the calculator more than doubled performance on all three math benchmarks, and the paper reports calculator use for 97.9% of examples across these benchmarks. arXiv
Open-domain question answering
For Web Questions, Natural Questions, and TriviaQA, the authors disabled the QA tool because the underlying QA system had been fine-tuned on Natural Questions, which would make part of the evaluation trivial. Instead, Toolformer mainly used Wikipedia search. The result was mixed: Toolformer clearly beat GPT-J-scale baselines, but it did not beat GPT-3-175B. arXiv
| Model | WebQS | NQ | TriviaQA |
|---|---|---|---|
| GPT-J | 18.5 | 12.8 | 43.9 |
| GPT-J + CC | 18.4 | 12.2 | 45.6 |
| Toolformer, tools disabled | 18.9 | 12.6 | 46.7 |
| Toolformer | 26.3 | 17.7 | 48.8 |
| OPT 66B | 18.6 | 11.4 | 45.7 |
| GPT-3 175B | 29.0 | 22.6 | 65.9 |
The authors attribute the remaining gap partly to the simplicity of the search engine and partly to Toolformer’s inability to interact with search results: it cannot browse, inspect multiple results, or reformulate a query when the first result is poor. This is a crucial boundary between Toolformer-style single-call tool use and ReAct/WebGPT-style interactive information seeking. arXiv
Translation and multilingual QA
Toolformer did not primarily evaluate translation quality with a translation benchmark such as BLEU. Instead, it evaluated multilingual question answering on MLQA, where the context paragraph is in English and the question may be in Spanish, German, Hindi, Vietnamese, Chinese, or Arabic. The machine-translation tool could translate the question into English, after which the model answered. arXiv
| Model | Es | De | Hi | Vi | Zh | Ar |
|---|---|---|---|---|---|---|
| GPT-J | 15.2 | 16.5 | 1.3 | 8.2 | 18.2 | 8.2 |
| Toolformer, tools disabled | 19.8 | 11.9 | 1.2 | 10.1 | 15.0 | 3.1 |
| Toolformer | 20.6 | 13.5 | 1.4 | 10.6 | 16.8 | 3.7 |
| GPT-3 175B | 3.4 | 1.1 | 0.1 | 1.7 | 17.7 | 0.1 |
Using the MT API improved Toolformer relative to Toolformer with tools disabled across all six reported languages, but Toolformer did not consistently beat vanilla GPT-J. The paper says further pretraining on CCNet deteriorated performance for some languages, and notes that the MT tool was used in 63.8% to 94.9% of examples depending on language, except Hindi, where usage was only 7.3%. arXiv
Temporal reasoning
For temporal grounding, Toolformer used a calendar API and was evaluated on TEMPLAMA and a new DATESET benchmark. Toolformer outperformed baselines on both, but the paper’s analysis is more nuanced than the table alone suggests. On TEMPLAMA, improvements mostly came from Wikipedia search and QA, not the calendar, because named entities often required factual lookup beyond merely knowing the date. On DATESET, the calendar tool mattered directly and was used for 54.8% of examples. arXiv
| Model | TEMPLAMA | DATESET |
|---|---|---|
| GPT-J | 13.7 | 3.9 |
| GPT-J + CC | 12.9 | 2.9 |
| Toolformer, tools disabled | 12.7 | 5.9 |
| Toolformer | 16.3 | 27.3 |
| OPT 66B | 14.5 | 1.3 |
| GPT-3 175B | 15.5 | 0.8 |
Language modeling preservation
One concern with tool-use fine-tuning is that the model might become worse as a general language model. Toolformer’s paper checks this by evaluating perplexity with tools disabled on WikiText and a held-out CCNet subset. The reported perplexities for GPT-J + CC and Toolformer with tools disabled are identical in the table: 10.3 on WikiText and 10.5 on CCNet. The authors conclude that adding API calls did not impose a perplexity cost for ordinary language modeling when API calls were disabled. arXiv
Scaling behavior
Toolformer also tested smaller GPT-2-family models. The ability to make good use of tools did not appear uniformly across scales; the paper reports that tool leverage emerged around 775M parameters, with smaller models showing little gain from API calls except in the easier Wikipedia-search-heavy setting. This matters because Toolformer is often summarized as “models can teach themselves tools,” but the result is conditional: the model must already be competent enough to generate plausible calls and use returned text. arXiv
Toolformer vs. ReAct
Toolformer and ReAct are complementary, not redundant.
| Dimension | Toolformer | ReAct |
|---|---|---|
| Primary mechanism | Fine-tune on self-generated, utility-filtered API calls | Prompt model to interleave reasoning traces and actions |
| Training requirement | Requires data generation, tool execution, filtering, and fine-tuning | Can be used few-shot at inference |
| Tool-use style | Mostly single-step inserted calls | Multi-step reasoning/action trajectories |
| Strength | Internalizes when/how to call simple tools | Handles interactive decision-making and exception handling |
| Weakness | No chained or interactive tool use in the original method | Depends heavily on prompting and model competence |
| Historical role | Training recipe for self-supervised tool-call behavior | Prompting pattern for agentic reasoning + acting |
ReAct’s main abstraction is an interleaved trajectory: think, act, observe, think again. Its paper reports that this improves interpretability and performance on question answering, fact verification, and interactive decision-making; in ALFWorld and WebShop, ReAct outperformed imitation and reinforcement learning baselines by absolute success-rate gains of 34% and 10%, respectively, while using only one or two in-context examples. arXiv
Toolformer’s abstraction is instead an augmented language-modeling dataset. It does not require reasoning traces, and it does not implement a general agent loop. The model learns to insert tool calls as token spans inside otherwise ordinary text. That makes Toolformer closer to Function Calling as later implemented in APIs, while ReAct is closer to the runtime loop used by Agentic Tool Use systems.
Relationship to retrieval-augmented generation
Toolformer overlaps with Retrieval-Augmented Generation, but the two solve different parts of the external-knowledge problem.
Classic RAG combines a parametric generator with non-parametric memory, usually a retriever over documents, so that generated text can condition on retrieved evidence. Lewis et al.’s RAG paper framed this as a way to improve knowledge-intensive generation, update world knowledge, and provide provenance by combining pretrained parametric memory with an explicit dense vector index of Wikipedia. arXiv
Toolformer treats retrieval as one possible tool among several. Its Wikipedia search tool is not the whole system; it sits alongside a calculator, QA system, machine-translation model, and calendar. The model is trained not merely to consume retrieved passages but to decide when search is worth calling at all.
This distinction anticipates a later architectural split:
| Architecture | Who decides to retrieve or call a tool? | What is optimized? |
|---|---|---|
| Standard RAG | Usually external orchestration or always-on retrieval | Better answer grounding and access to non-parametric memory |
| Toolformer | The LM, through learned token generation | Lower next-token loss after useful tool outputs |
| Native function calling | The LM, guided by schema and API training | Structured call generation in an application loop |
| Agent frameworks | Planner/controller, often the LM itself | Multi-step task completion under tool/environment feedback |
Toolformer is therefore not a replacement for RAG. It is one way to teach a model selective tool access, including selective retrieval.
Relationship to native tool-calling APIs
Modern native tool-calling APIs solved much of Toolformer’s interface problem with explicit schemas and message protocols. OpenAI’s current function-calling documentation describes tools as functionality made available to the model, tool calls as model requests to use that functionality, and tool outputs as the application-provided results sent back to the model; its documented flow is: make a request with tools, receive a tool call, execute code, send the output back, and receive a final response or more tool calls. OpenAI Developers
OpenAI’s June 2023 function-calling release made this interface explicit for developers: functions could be described to GPT-4 and GPT-3.5 Turbo, and the model could choose to output a JSON object containing arguments conforming to a function signature. The release also noted safety risks from untrusted tool output and recommended user confirmation before actions with real-world impact. OpenAI
Anthropic’s Claude documentation similarly describes tool use as a system where Claude decides when to call a tool based on the user request and tool description, returns a structured call, and either the application or Anthropic’s infrastructure executes it. Google’s Gemini function-calling documentation likewise frames function calling as allowing the model to determine when to call functions and provide parameters for external actions or data access. Claude
The relationship is best understood this way:
| Problem | Toolformer answer | Native API answer |
|---|---|---|
| How does the model express a call? | Linearized token span like <API> QA(query) → result </API> |
Structured tool/function call object, usually JSON-like |
| Who executes the tool? | External runtime interrupts decoding after the call marker | Developer platform or application runtime executes the call |
| How does the model learn when to call? | Self-supervised fine-tuning on utility-filtered calls | Provider-side training, prompting, schema descriptions, and application evals |
| How are arguments constrained? | Mostly by examples and learned format | JSON Schema or explicit tool definitions |
| How are results returned? | Inserted into the causal token stream | Sent back as tool-result messages or content blocks |
Native tool-calling APIs industrialized the interface. Toolformer remains important because it addressed the learning problem: how to create demonstrations for when tool use is useful without hand-labeling a large dataset.
Relationship to the broader tool-use literature
Toolformer was not the first system to augment a language model with tools. It was a particularly clear self-supervised formulation inside a broader wave.
MRKL Systems proposed a modular neuro-symbolic architecture combining LMs with external knowledge sources and discrete reasoning modules. The emphasis was systems architecture: route between neural and symbolic experts rather than expect one monolithic LM to do everything. arXiv
WebGPT fine-tuned GPT-3 to use a text-based browsing environment for long-form question answering, trained from human demonstrations and optimized with human feedback. It required browsing traces and citations, making it an early example of tool-using LMs for factuality and evidence collection. arXiv
TALM proposed Tool Augmented Language Models using non-differentiable tools and iterative self-play from a few demonstrations, with results on knowledge-heavy QA and math. Toolformer explicitly identifies TALM as closely related but distinguishes itself by broader generality and by applying tool use in a language-modeling setting rather than only downstream fine-tuning. arXiv
Program-Aided Language Models delegated computation to a runtime such as a Python interpreter: the LM decomposes a problem into code, while the interpreter executes the solution. PAL showed that external execution can outperform much larger models on mathematical, symbolic, and algorithmic reasoning tasks. arXiv
Later work such as Gorilla and ToolLLM moved toward large-scale API use. Gorilla focused on accurate API-call generation and retrieval over changing documentation, motivated by the observation that even strong LLMs can hallucinate API usage or generate incorrect arguments. ToolLLM constructed ToolBench from 16,464 real-world RESTful APIs across 49 categories and used ChatGPT-generated instructions and solution paths to train and evaluate tool-using models. arXiv
API-Bank made the evaluation problem explicit: it introduced a runnable benchmark with 73 API tools and 314 annotated tool-use dialogues, and its authors emphasized that major questions remained about how effective LLMs are at tool use, how to improve them, and what obstacles remain. ACL Anthology
The pattern across this literature is that tool use has at least four separable problems: interface, retrieval, planning, and learning. Toolformer made its strongest contribution to learning.
Limitations
1. The utility filter is only as good as next-token loss
Toolformer’s filter keeps calls that reduce next-token loss. That is elegant, but it is not identical to task success, truthfulness, safety, or long-horizon utility. A tool output may make a local continuation more predictable while still being misleading, irrelevant, or harmful in a downstream task.
The paper’s own qualitative analysis notes that high utility scores often correspond to useful calls but not always; noisy calls can survive filtering. This is not a minor implementation issue. It is the cost of replacing human judgment with a proxy objective. arXiv
2. The tool set is narrow
The tools in Toolformer are text-in/text-out and relatively simple. The calculator supports only the four basic arithmetic operations. The calendar returns a date. Search returns snippets. The MT tool translates into English. This is enough to test the method, but it is far from arbitrary real-world API use involving authentication, side effects, mutable state, long JSON schemas, error handling, streaming outputs, or irreversible actions. arXiv
3. No chained tool use
The paper explicitly states that Toolformer cannot use tools in a chain, meaning it cannot take the output of one tool and use it as the input to another. This follows directly from the data-generation process: API calls for each tool are generated independently, so the fine-tuning corpus does not contain chained tool-use examples. arXiv
This limitation is especially damaging for agentic workflows. Many real tasks require chains such as: search for a source, extract a date, calculate a deadline, query a database, send a message, and verify completion. Toolformer’s original method does not train that behavior.
4. No interactive search or browsing
Toolformer can call search, but it cannot browse through multiple results, reformulate a query, or inspect a page iteratively. The authors explicitly identify this as a limitation, especially for search engines that may return many possible results. This is where ReAct and WebGPT point in a different direction: multi-step interaction with an environment. arXiv+2arXiv+2
5. Sensitivity and sample inefficiency
The paper reports that Toolformer can be sensitive to exact input wording when deciding whether to call an API. It also reports sample inefficiency: processing more than a million documents can yield only a few thousand useful calculator examples. The method is therefore not a free lunch; it replaces human labeling with compute, heuristic filtering, and dependence on the model’s ability to generate plausible candidates. arXiv
6. No cost-aware objective
Toolformer’s filter asks whether a tool call improves next-token prediction. It does not ask whether the improvement is worth the tool’s latency, monetary cost, risk, or side effects. The authors explicitly note that the method does not account for tool-dependent computational cost when deciding whether to make an API call. arXiv
Modern production systems cannot ignore this. A web search, database query, code execution, refund API, or email-sending tool has very different cost and risk from a local calculator.
7. Out-of-distribution tool brittleness
Toolformer’s behavior is learned from generated examples for a fixed set of tools. The paper does not establish that the model can robustly generalize to new tools described only at inference time, to large schema spaces, or to tools whose semantics differ from the training distribution.
Later work reinforces the difficulty. Gorilla’s authors motivate their API-focused model by noting that even strong LLMs struggle with accurate input arguments and hallucinated API usage; ToolLLM and API-Bank similarly treat broad real-world API use as a separate training and evaluation problem rather than something solved by early tool-use methods. arXiv+2arXiv+2
The open question: does self-supervised tool-use learning scale to agents?
Toolformer answers a local question: can a model learn single-step tool insertion from self-generated examples filtered by next-token utility? Yes, under constrained tools and tasks.
It does not answer the larger question: can self-supervised tool-use learning produce robust AI Agents that plan, call many tools, recover from errors, manage state, respect cost constraints, and operate safely in open-ended environments?
The evidence remains thin. The original paper’s limitations rule out chaining and interaction. ReAct demonstrates that prompting can produce multi-step reasoning/action trajectories, but that is not the same as Toolformer-style self-supervised training. API-Bank and ToolLLM show that tool-use evaluation and training at larger API scale require explicit datasets, solution paths, retrieval, and evaluators. arXiv+3arXiv+3arXiv+3
The deeper issue is credit assignment. Toolformer’s loss filter works because a useful tool call often helps predict nearby tokens. Agentic tasks often have delayed, sparse, non-linguistic rewards. Sending the correct email, editing the correct file, booking the correct appointment, or debugging a program may not correspond to local next-token likelihood. A model may need to explore, backtrack, verify, and choose actions whose value is visible only after many steps.
A 2023 survey of augmented language models framed fully self-supervised meaningful augmentation as an open research question and emphasized that few works combined reasoning and tools; it also warned that tool-augmented models can appear more authoritative while still being wrong, and that active tools broaden the space of possible harms. arXiv
The open research question is therefore not whether tools help LMs. That is settled. The open question is whether a Toolformer-like objective can scale from local predictive utility to long-horizon agentic utility.
Design lessons for AI engineering
Toolformer still offers several durable engineering lessons.
First, tool use should be selective. Always-on retrieval or always-on tool execution can waste context, latency, and money. Toolformer’s learned call policy is an early example of making tool access conditional.
Second, tool-use training data can be generated automatically, but it needs a filter. The paper’s most reusable idea is not the exact API syntax; it is the generate-execute-filter pipeline. Modern synthetic-data pipelines for agents often follow the same broad shape, even when the filter is a verifier, unit test, evaluator model, environment reward, or human preference model rather than next-token loss.
Third, tool outputs should be treated as part of the model’s context, not as magical ground truth. Toolformer learns to incorporate returned text, but the paper’s limitations and later API safety guidance both show that tool outputs can be noisy, adversarial, or irrelevant. arXiv
Fourth, explicit interfaces matter. Toolformer used linearized API tokens because it operated inside a plain language-modeling setup. Native function-calling APIs replaced that with structured schemas and tool-result messages, making application development more reliable. But the existence of a schema does not solve tool choice, argument quality, verification, or safety. OpenAI Developers+2Claude+2
Finally, tool-use competence is not monolithic. A model can be good at calculator calls and poor at search. It can improve multilingual QA through translation without consistently beating a baseline. It can learn to call tools only after reaching sufficient scale. Any serious evaluation must separate tool selection, argument construction, execution handling, result interpretation, and final answer quality. Toolformer’s ablations make this decomposition visible. arXiv
Bottom line
Toolformer is best read as a proof of concept for Utility-Filtered Self-Supervision in tool-using language models. It showed that a pretrained LM can bootstrap tool-use demonstrations from raw text, filter those demonstrations by predictive usefulness, and internalize a policy for calling simple tools.
Its limitations are equally important. It is not a general agent architecture, not a complete solution to arbitrary API use, and not a substitute for interactive planning, verification, or safety controls. Its lasting contribution is the idea that tool-use data can be produced by the model itself and selected by utility — an idea that remains central to self-improving systems, synthetic data generation, and the search for scalable supervision.
References
[Toolformer paper] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom, “Toolformer: Language Models Can Teach Themselves to Use Tools,” 2023. arXiv
[ReAct paper] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao, “ReAct: Synergizing Reasoning and Acting in Language Models,” 2022/2023. arXiv
[RAG paper] Patrick Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” 2020/2021. arXiv
[MRKL Systems] Ehud Karpas et al., “MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning,” 2022. arXiv
[WebGPT paper] Reiichiro Nakano et al., “WebGPT: Browser-assisted question-answering with human feedback,” 2021/2022. arXiv
[TALM paper] Aaron Parisi, Yao Zhao, and Noah Fiedel, “TALM: Tool Augmented Language Models,” 2022. arXiv
[PAL paper] Luyu Gao et al., “PAL: Program-aided Language Models,” 2022/2023. arXiv
[Gorilla paper] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez, “Gorilla: Large Language Model Connected with Massive APIs,” 2023. arXiv
[ToolLLM paper] Yujia Qin et al., “ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs,” 2023/2024. arXiv
[API-Bank paper] Minghao Li et al., “API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs,” EMNLP 2023. ACL Anthology
[Augmented Language Models survey] Grégoire Mialon et al., “Augmented Language Models: a Survey,” 2023. arXiv
[OpenAI function-calling release] OpenAI, “Function calling and other API updates,” June 2023. OpenAI
[OpenAI function-calling docs] OpenAI API documentation, “Function calling.” OpenAI Developers
[Anthropic tool-use docs] Anthropic Claude API documentation, “Tool use with Claude.” Claude
[Gemini function-calling docs] Google AI for Developers, “Function calling with the Gemini API.” Google AI for Developers
Companion entries
Core theory: Tool Use in LLMs, Self-Supervised Learning, Utility-Filtered Self-Supervision, Next-Token Prediction, Augmented Language Models, Neuro-Symbolic AI
Adjacent systems: ReAct, MRKL Systems, WebGPT, Program-Aided Language Models, TALM, Gorilla, ToolLLM
Retrieval and grounding: Retrieval-Augmented Generation, Selective Retrieval, Non-Parametric Memory, Wikipedia Search as a Tool, Factuality in Language Models
API infrastructure: Native Tool Calling APIs, Function Calling, Structured Outputs, Tool Routing, Model Context Protocol, Agent Runtime Loops
Evaluation and safety: API-Bank, Agent Benchmarks, Tool-Use Reliability, Prompt Injection, Human-in-the-Loop Authorization, Cost-Aware Tool Use
Open questions: Agentic Tool Use, Self-Improving Systems, Tool Chaining, Long-Horizon Credit Assignment, OOD Generalization in Agents