Coverage note: this entry is updated through April 4, 2026.
Overview
The cleanest verdict through early 2026 is that Sutton's core asymptotic claim aged well. Since 2019, the biggest jumps at the capability frontier have mostly come from scalable general methods: large pretrained transformers, compute-optimal scaling recipes, reinforcement-learning-enhanced reasoning, test-time search, and diffusion models. Kaplan-style scaling laws, Chinchilla's training corrections, GPT-4's predictability from smaller runs, the o-series' gains from extra reasoning compute, and the GPT-5 family's routed general system all fit the same broad pattern: frontier progress came less from hand-built domain theories than from methods that keep getting better when given more compute, more data, or more search. (Incomplete Ideas)
But the essay was also overread. The strongest surviving rebuttal is not "Sutton was wrong." It is that "The Bitter Lesson" is a statement about long-run drivers of capability, not a complete recipe for building reliable or economical systems. Data limits are real, reasoning can be expensive, public reasoning systems still perform poorly on tasks like ARC-AGI-2, and deployed systems in 2026 are usually hybrids: large general models wrapped in routing, tools, retrieval, structured constraints, evals, and human oversight. Sutton himself now argues that current LLMs may only partially exemplify the lesson because they learn from static human text rather than open-ended experience. (arXiv)
Where evidence below comes from official vendor reports, it is best treated as suggestive rather than final. Those reports matter because they describe the systems that actually shaped the frontier, but they are not a substitute for independent replication.
1. Sutton's original argument
Sutton's 2019 essay is short enough that many later readers projected more onto it than it actually said. The text's essential claim is that the "biggest lesson" of AI history is that "general methods that leverage computation" win in the long run, and that the two methods with the deepest scaling potential are "search and learning." The target of the essay was not all structure, all engineering, or all human judgment. The target was the recurrent AI habit of baking in our current understanding of a domain so deeply that systems improve quickly for a while and then stop scaling. (Incomplete Ideas)
That matters because the essay is often paraphrased as "scale is all you need," which is stronger than Sutton's text. Sutton did not say parameter count alone is destiny. He said methods should be chosen for how well they convert future compute into future performance. In 2025, Sutton himself complicated the popular LLM-centric reading: he argued that LLMs are only "in some ways" a classic case of the Bitter Lesson and may ultimately be superseded by systems that learn directly from experience rather than only from human-produced corpora. (Incomplete Ideas)
The best 2026 reading of the essay is summarized below. It synthesizes Sutton's text with later clarifications and the way modern systems are actually built. (Incomplete Ideas)
| Suttonian proposition | Best 2026 reading | Bad overreading |
|---|---|---|
| General methods leveraging computation win in the long run | Prefer approaches whose performance improves smoothly with more data, compute, or search | "Just increase parameter count" |
| Search and learning are the deepest scalable levers | Train-time learning and inference-time search/test-time compute both matter | "Next-token pretraining is the final algorithm" |
| Hard-coding human domain knowledge is fragile | Use human structure sparingly and in ways that do not cap scaling | "All engineering, tools, routing, and constraints are anti-Sutton" |
2. Reception: from manifesto to Rorschach test
From 2019 to 2023, "The Bitter Lesson" moved from influential essay to ideological shorthand. GPT-4 accelerated that shift. OpenAI's technical report described a large multimodal transformer whose performance on some measures could be forecast from models trained with at most one-thousandth of GPT-4's compute. Microsoft's "Sparks of AGI" paper then read early GPT-4 as showing striking breadth across law, medicine, coding, psychology, visual tasks, and mathematics. To the pro-scaling camp, this looked like Sutton's historical thesis turning into live engineering fact. (arXiv)
Critics pushed back on two fronts. First, they argued that the field had silently replaced Sutton's narrower claim with a stronger one. Chollet's "On the Measure of Intelligence" argued that unlimited priors or unlimited training data can buy arbitrarily high task skill while obscuring a system's actual efficiency at acquiring new skills. On that view, the GPT-era discourse risked conflating benchmark skill with general intelligence. (arXiv)
Second, critics argued that scale had improved fluency faster than trustworthiness. Marcus's 2022 critique said bigger GPT-like systems had become more articulate without achieving the comprehension, truthfulness, or reliability needed for robust AI. That criticism was not purely anti-Sutton; Marcus later called the essay "truly great" while also describing it as overstated and flawed. That combination is important. Even many critics of scale maximalism accepted that Sutton had identified a real historical force; they denied only that it settled the problem of intelligence. (Nautilus)
By 2025, the essay had become a Rorschach test for the whole field. Pro-scaling researchers read it as permission to keep pushing general models with more compute. ARC-style skeptics read it as a warning not to mistake broad skill for fluid abstraction. And Sutton himself partly exited the LLM-as-final-form reading, arguing that systems trained on human text might hit data ceilings and that the more Bitter-Lesson-consistent future would involve learning from experience in open environments. (ARC Prize)
3. The strongest supporting evidence since 2019
3.1 Scaling laws made the thesis quantitative
Kaplan et al. did more than add another good language model. They made a central Suttonian intuition measurable. Their paper found that language-model loss follows power-law relationships with model size, dataset size, and training compute across enormous ranges, while width/depth details matter comparatively little within a broad regime. The immediate implication was not merely "bigger is better," but "there exist regular scaling regimes in which general methods improve predictably." That is exactly the kind of landscape in which domain-specific cleverness tends to lose to smooth scaling. (arXiv)
Chinchilla sharpened, rather than weakened, the Bitter Lesson. Hoffmann et al. showed that many frontier models were undertrained: the best use of a fixed compute budget was not maximum parameter count but a better balance of model size and tokens. Their compute-optimal result—roughly doubling both model size and data together—produced a 70B model that outperformed much larger systems and also reduced fine-tuning and inference cost. This is a direct rebuttal to the naïve version of scale talk while remaining fully consistent with Sutton's deeper point. The winning move was still to improve a general scaling recipe, not to replace it with handcrafted domain structure. (arXiv)
The subtle but important update is this: Sutton's lesson survived the transition from "bigger models" to "better scaling regimes." In other words, Chinchilla did not rescue artisanal AI from scale; it rescued scale from its own first crude formulation. (arXiv)
3.2 GPT-4, reasoning models, and the GPT-5 family
GPT-4 was strong evidence for the Bitter Lesson not merely because it was large, but because it was predictable. OpenAI reported that some aspects of GPT-4's performance could be accurately extrapolated from much smaller training runs. That matters because predictable scaling is the opposite of brittle feature engineering: it means performance is being driven by a regular compute-governed regime rather than by a pile of local tricks. (arXiv)
The reasoning-model wave strengthened the case further by vindicating the second half of Sutton's pair: search. OpenAI's o1 report showed large jumps over GPT-4o on AIME, with still larger gains when the model used consensus or re-ranking over many samples. OpenAI's o3 announcement then made the point explicitly: large-scale RL showed the same "more compute = better performance" pattern in reasoning that GPT pretraining had shown earlier, and performance kept improving when the model was allowed to think longer. That is an unusually direct post-2019 confirmation of Sutton's claim that scalable search and learning are the deep long-run levers. (OpenAI)
The public ARC-AGI story is especially revealing here. ARC Prize reported that o3-preview achieved 75.7% on ARC-AGI-1's semi-private evaluation under its public compute cap, and 87.5% in a much higher-compute setting. Even if one rejects ARC-AGI-1 as decisive evidence for general intelligence, the result is still a clean demonstration that more inference-time search/test-time compute can unlock large additional capability in a general reasoning system. Sutton's "search" term looks much less historical after results like that. (ARC Prize)
GPT-5 extended the pattern from research to product architecture. OpenAI introduced GPT-5 as a unified system consisting of a fast model, a deeper reasoning model, and a router trained to decide which to use. Current 2026 docs describe GPT-5.4 as OpenAI's default frontier model for both general-purpose work and most coding tasks, with gains in tool use, long-running workflows, multimodal perception, and agentic web search, plus a 1M-token context window and built-in computer use. A simplistic reading says this weakens the original scale story because the frontier system is no longer one monolithic model. A better reading says the opposite: the frontier is still being pushed by general learning systems that allocate computation flexibly, not by a return to brittle domain-specific software. (OpenAI)
OpenAI's GPT-5 benchmark claims should be read cautiously because they are vendor-authored. But even under that caveat, the direction of travel is hard to miss: frontier capability continued to come from general systems with more compute, more routing, more tool use, and more reasoning, not from injecting handcrafted theories of law, medicine, or software engineering into separate bespoke engines. (OpenAI)
3.3 Diffusion models are the parallel confirmation outside language
The cleanest non-LLM support for Sutton comes from generative vision. DDPMs achieved high-quality image synthesis with a generic denoising objective rather than GAN-style adversarial trick engineering. That was already a Bitter-Lesson-shaped move: use a broad learning procedure that can absorb more data and compute instead of fine-tuning hand-designed pipelines. (arXiv)
Latent Diffusion adds an important nuance. Rombach et al. showed that diffusion in latent space dramatically reduced compute while preserving quality and general conditioning flexibility. This did not refute the Bitter Lesson. It showed how a small amount of architecture-level structure can make a general method economically scalable. In retrospect, this is a useful template for understanding the 2026 frontier more broadly: minimal structure around a general scalable core often beats both extremes of "pure brute force" and "heavy domain engineering." (arXiv)
Diffusion Transformers pushed the same story into transformer scaling. Peebles and Xie found that DiTs with higher Gflops consistently achieved better FID, and later work reported explicit power-law scaling laws for DiT pretraining and generation quality. That is Bitter-Lesson logic almost verbatim: performance improves predictably as the system is given more compute, while the core method remains broad and reusable. (arXiv)
Taken together, scaling laws, GPT-4/5, reasoning models, and diffusion models make the pro-Sutton case very strong. If one asks where frontier performance gains actually came from after 2019, the answer is overwhelmingly: more scalable learning, better compute allocation, and revived forms of search/test-time computation. (arXiv)
4. The strongest counterarguments
4.1 The data wall is real, even if the hardest version is not yet proven
The strongest resource-level objection is that the easy era of internet-scale imitation may be finite. Villalobos et al. estimate that, if existing LLM trends continue, training datasets could approach the stock of public human text between 2026 and 2032. That is not a proof of imminent stagnation, but it is a serious constraint on the simple "just keep scaling pretraining" story. (arXiv)
Model-collapse work sharpened the concern. Shumailov et al. argued that recursively training models on model-generated data can degrade the tails of the distribution and eventually produce a distorted, lower-variance world model. That sounds like a direct challenge to the Bitter Lesson if future scale depends increasingly on synthetic data. But follow-up work made the picture less fatalistic: Gerstgrasser et al. found that collapse occurs when synthetic data replace the original real distribution, whereas accumulating synthetic data alongside original real data can avoid the collapse dynamic they studied. The evidence here is genuinely mixed. The data wall is a live systems problem; the claim that it permanently breaks scaling is still unproven. (Nature)
Sutton's own later comments fit that nuance. His 2025 argument is not that scaling ends, but that this particular substrate—static imitation of human text—may not be the final scalable path. On his telling, the next Bitter-Lesson-compliant step would be systems that gather data by acting in the world and learning from experience. That is not a rejection of the lesson. It is a relocation of where future scale would have to come from. (Dwarkesh)
4.2 Inference cost and latency are not side issues
A second rebuttal is economic rather than conceptual. Sutton's essay is about what wins in the long run, but practitioners live in the short and medium run of latency budgets, API bills, and user patience. ARC Prize's o3-preview results dramatized this: the model's higher-compute configuration reached 87.5% on the semi-private ARC-AGI-1 evaluation, but at an estimated total benchmark cost of $456,000, versus a far cheaper high-efficiency setup. That does not disprove capability scaling, but it does show how easily "frontier possible" diverges from "product viable." (ARC Prize)
The same tension appears in ordinary deployment. OpenAI's current pricing shows that frontier reasoning models and tool use remain meaningfully expensive, especially for long contexts or tool-heavy workflows. GPT-5.4 is much cheaper than previous frontier regimes on a capability-per-dollar basis, but it is not remotely "free compute." For many businesses, the relevant question is not whether the most general method can eventually win, but whether it beats specialized alternatives at a tolerable cost today. (OpenAI)
That said, the cost objection is only a partial rebuttal because price/performance has also improved rapidly. Epoch reports that inference prices for fixed capability milestones have fallen dramatically, albeit unevenly, and OpenAI says GPT-5 gets better results than o3 with 50–80% fewer output tokens on several evaluations. So the anti-Sutton version—"inference cost will permanently block general methods"—looks too strong. The right conclusion is narrower: economics can delay or redirect the Bitter Lesson even when it does not falsify it. (Epoch AI)
4.3 "Skill is not intelligence" remains a live objection
Chollet's critique is still among the strongest conceptual counterarguments because it attacks the benchmark logic, not any one model. If unlimited data and priors let you buy task skill, then large improvements on standard evaluations do not automatically show efficient, general abstraction. ARC-AGI was designed to operationalize that criticism. (arXiv)
ARC Prize's own 2025 materials underline the point. The foundation reported that pure LLMs score 0% on ARC-AGI-2, public reasoning systems only reach single digits, and even o3/o4-mini low or medium settings stayed under 3% on ARC-AGI-2 while doing much better on ARC-AGI-1. That is not evidence that modern models are weak overall; it is evidence that high average benchmark performance does not settle the deeper question of fluid, efficient generalization. (ARC Prize)
This is one of the strongest anti-maximalist points in the whole debate. Sutton's thesis is about the long-run superiority of scalable general methods over bespoke engineering. It says nothing by itself about which benchmark best measures intelligence, or whether current scaling regimes are already capturing the kind of generalization humans care about. Chollet's critique and ARC-AGI keep that question open. (arXiv)
4.4 Reliability, world models, and trust never reduced to "just scale"
Marcus's main criticism was not that scale produces no gains. It was that fluency, factuality, and trustworthy reasoning do not automatically improve at the same rate, and that some kinds of explicit structure may still be indispensable. In 2022 he argued that larger GPT-style models had become more fluent without becoming proportionally more trustworthy or grounded. (Nautilus)
Some of the sharpest versions of that critique have weakened. OpenAI now reports substantially lower hallucination and deception rates for GPT-5 relative to GPT-4o and o3, and greater factual reliability on open-ended fact-seeking tasks. But this does not settle the objection, partly because the evidence is vendor-authored, and partly because the vendors' own deployment advice still assumes raw model competence is insufficient. OpenAI recommends guardrails, structured tools, evals, and human intervention; Anthropic makes similar recommendations and warns that context is finite and must be engineered carefully. If raw next-token competence were already enough, these layers would be less central than they now are. (OpenAI)
Sutton's own later comments again intensify the nuance. In his 2025 interview, he argued that LLMs predict what people would say rather than what will happen, and that truly scalable intelligence should learn from experience with goals and consequences. That is not Marcus's neurosymbolic argument, but it converges on the same basic point: the current LLM stack may not yet embody the strongest possible version of the Bitter Lesson. (Dwarkesh)
4.5 Domain-specific specialization still buys real gains
The public evidence for "domain-specific fine-tuning beats generality" is weaker than the evidence for scaling laws, but the practical case is real. OpenAI's own guidance for agents is to prototype with the most capable model, establish a baseline, and then replace pieces of the workflow with smaller models where cost and latency allow. Anthropic's long-running-agent work similarly says it remains an open question whether a single general-purpose coding agent is best, or whether specialized testing, QA, or cleanup agents perform better on sub-tasks. The strongest 2026 case for specialization is therefore operational, not theoretical: general models dominate the capability frontier, while specialized adaptations often dominate the cost/reliability frontier. (OpenAI)
In formal domains, engineered structure matters even more. AlphaProof and AlphaGeometry achieved medal-level mathematical performance by combining RL, formal environments, and domain-specific representations. Their combined 2024 system solved four of six IMO problems, but the pipeline required experts to translate problems into formal languages and spent two to three days of computation on some solutions. In 2025, DeepMind reported that an advanced Gemini Deep Think system reached gold-medal standard end-to-end in natural language within competition time, but it still used a specialized reasoning mode with parallel thinking. These are spectacular achievements. They are also not examples of "one generic chat model plus more parameters solves formal math." (Nature)
The symbolic counterexample is real, too. A separate 2024 paper revisiting AlphaGeometry's benchmark found that classical Wu's method, combined with other symbolic geometry techniques, could solve 21/30 problems on a CPU-only laptop within five minutes per problem—just four fewer than AlphaGeometry—and could complement AlphaGeometry to set a higher combined state of the art. This does not mean symbolic methods have "won." It means domain structure can still be highly leverageable in exact, verifiable regimes. (arXiv)
4.6 Agent scaffolding is not a sideshow
The strongest practical counterargument to scale maximalism is that system design matters a great deal. Anthropic says successful agent deployments generally use simple, composable patterns rather than baroque frameworks, and its own long-running-agent research still treats single-agent versus multi-agent specialization as an open problem. OpenAI's practical guide says much the same: start with a single capable agent, add tools and structure, and only move to multi-agent orchestration when simpler designs stop working. (Anthropic)
SWE-bench Verified makes the same point at benchmark level. The benchmark explicitly separates "all agents" from an apples-to-apples bash-only setting using mini-SWE-agent, a minimal ReAct loop without elaborate tools or scaffold structure, and its maintainers report strong results from a roughly 100-line implementation. At the same time, OpenAI reports that o3 reaches new SWE-bench SOTA without a custom model-specific scaffold. The honest synthesis is therefore two-sided: scaffolds matter a lot, and base-model quality matters a lot. Anyone who explains 2025–2026 coding progress using only one of those variables is throwing away half the picture. (SWE-bench)
5. The "jagged frontier" nuance
The best short label for the 2026 nuance is the jagged technological frontier. In experimental work on knowledge workers, Brynjolfsson, Li, and Raymond showed that AI improves performance on some tasks but degrades it on others, including tasks that seem similarly difficult to humans. The Organization Science version makes the result blunt: within the frontier, AI complements human work; outside it, AI output can be inaccurate enough to hurt performance. This is the concept that best explains why both the Bitter-Lesson camp and its critics can sound right at the same time. (PubsOnline)
The table below condenses that jagged-frontier view using the main sources in this article: scaling-law papers, ARC results, formal-math systems, and current deployment guides. (arXiv)
| Region of the frontier | What tends to win in 2026 | Why |
|---|---|---|
| Broad text, code, and multimodal tasks with abundant data | Large general models plus tools | Weak regularities are everywhere, and scale exploits them well |
| Reasoning tasks with cheap verification and lots of test-time compute | General models plus RL and search | Extra computation buys measurable gains |
| Formal, exact, or highly structured domains | Hybrids with formal systems, symbolic structure, or explicit constraints | Correctness and search-space control matter more than average fluency |
| Long-horizon or high-stakes workflows | Hybrid systems with routing, context engineering, guardrails, and human oversight | Memory limits, error accumulation, and irreversible actions dominate |
This is where the modern debate becomes less tribal. The "general methods" side is strongest where data are abundant, feedback is cheap, and mistakes are tolerable enough to search over. The "engineered structure" side is strongest where feedback is sparse, correctness is exacting, or costs compound over long horizons. That is not a contradiction of Sutton. It is the missing practical qualifier. (PubsOnline)
6. How practitioners actually build systems in 2026
If one sentence summarizes 2026 practice, it is: frontier model at the core, engineered system around it. OpenAI's agent guide says to start with the most capable model to establish a quality baseline, then substitute smaller models where they still meet the target. Anthropic's agent guidance says the most successful teams usually rely on simple patterns rather than ornate frameworks, and its context-engineering note argues that managing context—not just prompt wording—has become a central engineering problem. In other words, practitioners are not choosing between "general models" and "engineering." They are wrapping general models in engineering. (OpenAI)
The architecture that follows is strikingly convergent across organizations. A strong base model handles broad cognition. Retrieval, tools, or browsing fetch external facts and state. Structured prompts or policy templates constrain behavior. A router or subagents may split work when needed. Guardrails, evaluations, and human intervention protect high-risk actions. GPT-5 itself is publicly described this way—as a routed system rather than a single pure model—and OpenAI's practical guide treats human escalation as a core reliability mechanism. Anthropic's long-running-agent work similarly emphasizes harness design, progress tracking, self-verification, and explicit handling of recurrent failure modes. (OpenAI)
Coding practice in 2026 is particularly revealing because it forces the economics and reliability questions into the open. Anthropic reports that developers use AI in roughly 60% of their work but can "fully delegate" only 0–20% of tasks; the rest still requires setup, supervision, validation, and judgment. Thoughtworks reports a complementary "80/20" pattern from practitioner conversations: roughly 80% of software can be generated with automated verification, while the highest-risk or highest-value 20% still gets deep human attention. This is not the abolition of software engineering. It is a redistribution of where the rigor lives. (Anthropic Resources)
There is also a more aggressive, frontier-user version of the 80/20 ratio. Media reports on Andrej Karpathy's March 2026 remarks describe his personal workflow as having flipped from roughly 80% manual coding / 20% agents to the reverse, with most of his work becoming English-language instruction, review, and touch-up rather than direct code entry. That should be treated as anecdotal evidence about an elite early adopter, not as an industry base rate. But it matters because it shows where the practice frontier had moved by early 2026. (Business Insider)
Karpathy's other comments reinforce the same hybrid reading. He argues this is the decade, not the year, of agents because current systems still lack enough intelligence, multimodality, computer use, continual learning, and robustness to function like true employees. That is exactly the kind of diagnosis one would expect if general methods are dominant at the frontier but still not sufficient, on their own, to collapse all the old systems-engineering problems. (Dwarkesh)
Benchmark practice now mirrors production practice. SWE-bench Verified exists partly to disentangle raw model capability from scaffold advantage; the bash-only mini-SWE-agent evaluations do that by fixing the harness. But full systems still outperform the bare loop in many cases, and vendors increasingly optimize models for tool use, long-running tasks, and multi-step workflows directly. By 2026, "model intelligence" and "system intelligence" are no longer cleanly separable. The winners improve both. (SWE-bench)
7. Assessment
As a theory of where frontier capability gains have come from since 2019, The Bitter Lesson has mostly won. The field's decisive progress was not driven by hard-coded ontologies, symbolic grammars for every problem, or narrow expert systems for each domain. It was driven by broad models, more compute, better compute allocation, large-scale learning, and the return of search at inference time. On that question, Sutton's essay looks prescient rather than merely provocative. (Incomplete Ideas)
What the essay did not settle is whether current LLM-style pretraining is the final scalable substrate, whether scale alone produces trustworthy or efficient intelligence, or whether a single giant model should dominate every practical deployment. The surviving counterarguments—data limits, inference cost, ARC-style efficiency gaps, formal-domain structure, scaffolding effects, and long-horizon agent brittleness—are all real. None of them has yet overturned Sutton's asymptotic thesis. All of them constrain its practical meaning. (arXiv)
The best 2026 restatement is therefore not "scale is all you need." It is closer to this: start from the most general scalable method you can afford, then add only the structure that buys reliability, data efficiency, or cost without freezing future scale. That formulation fits Chinchilla, diffusion, reasoning models, GPT-5 routing, ARC failures, formal-math hybrids, and real-world agent engineering far better than either camp's slogan. (arXiv)
The essay remains "bitter" for one reason above all: it demotes human cleverness from the center of the story. The 2026 update is that human cleverness did not disappear; it moved up a level. We now hand-design routers, tools, harnesses, evals, and verification layers around very general models. The craftsmanship shifted from encoding knowledge inside the model to engineering the environment in which scalable models learn, reason, and act. (OpenAI)
Companion entries
Core theory: Scaling Laws, Chinchilla Scaling, Search and Learning, Reward is Enough, Experience vs Imitation in AI
Frontier evidence: GPT-4 Technical Report, GPT-5, Reasoning Models, Test-Time Compute, Diffusion Models, Diffusion Transformers
Counterarguments: ARC-AGI, The Data Wall, Model Collapse, Skill vs Intelligence, Inference Cost Burden
Practice and systems: Jagged Technological Frontier, Agent Scaffolding, Context Engineering, Human-in-the-Loop AI, AI Coding Agents, AlphaProof, AlphaGeometry