Self-improving software did not begin with deep learning, and most of it has never looked like the science-fiction image of a machine recursively redesigning itself into open-ended superiority. The real historical lineage from Doug Lenat's EURISKO to contemporary LLM agents is more concrete: expose some part of the system as a mutable object, score candidate modifications against a feedback signal, keep the winners, and repeat. What changed over four decades was the exposed substrate. In the early 1980s it was symbolic heuristics; in the 1990s it was executable program trees; in the 2010s it became ML pipelines and neural architectures; by the mid-2020s it had shifted upward again into prompts, tool interfaces, skill files, configuration manifests, and evaluation harnesses wrapped around largely fixed foundation models. (Northwestern CS)
The strongest historical pattern is not "recursive self-improvement" in the maximal sense. It is a sequence of increasingly engineered outer loops. Each era found a better answer to the same question: what part of the system is cheap enough, structured enough, and legible enough to optimize automatically? Each era also ran into the same four ceilings: representation mismatch, evaluator weakness, search cost, and overfitting to the metric. That is why the history is better told as a lineage of constrained self-modification than as a march toward unbounded autonomy. (Paraprogramming Dispatches)
Historical thesis: self-improvement became more practical as the mutable substrate became narrower, more modular, and more explicitly testable. EURISKO was broad and conceptually ambitious but brittle; AutoML and NAS were much narrower but far more reproducible; modern agents are broader again, but mostly because the mutable objects are now ordinary software artifacts—files, manifests, tool descriptions, and evals—that can be versioned and re-scored. (Northwestern CS)
At a glance
| Era | Representative systems | Self-improvement mechanism | What was modified | Feedback signal | Main brake |
|---|---|---|---|---|---|
| 1980s symbolic discovery | EURISKO (Northwestern CS) | Higher-order heuristics invent, prune, and revise other heuristics | Heuristic frames, slot types, representation vocabulary, agenda operators | Heuristic "worth," agenda success, domain outcomes, human curation | Representation mismatch, sparse knowledge, human-in-the-loop |
| 1990s evolutionary program search | Genetic programming; ADFs; architecture-altering operations (Genetic Programming) | Variation and selection over populations of executable programs | Program trees, subroutines, parts of program architecture | Task fitness on benchmark cases | Fitness engineering, evaluation cost, code bloat |
| 2010s pipeline search | Auto-WEKA; auto-sklearn (arXiv) | Bayesian optimization over structured ML pipeline spaces | Algorithm choice, preprocessing, hyperparameters, ensembles | Validation loss under compute budget | Meta-overfitting to validation, bounded search spaces |
| Late-2010s neural design search | NAS; AmoebaNet; ENAS; DARTS (arXiv) | RL, evolution, or gradient search over architecture graphs | Network architecture; sometimes shared search-time weights | Validation accuracy / perplexity | Massive compute, ranking instability, reproducibility problems |
| 2023–2026 agent scaffolding search | Self-Refine; Reflexion; Voyager; Promptbreeder; LATM; ToolMaker (arXiv) | Reflection, prompt evolution, skill accumulation, tool creation | Prompts, memory text, skill libraries, tool code, tool descriptions | Task reward, tests, environment feedback, grader outputs | Short horizons, tool brittleness, evaluator quality |
| 2025–2026 self-editing coding agents | SICA; Darwin Gödel Machine; Huxley-Gödel Machine; Live-SWE-agent (arXiv) | Agents rewrite their own code/scaffold, then re-evaluate | Agent codebase, tools, context management, peer review logic, scaffold files | Coding benchmarks and held-out tasks | Benchmark mismatch, thin evidence base, sandbox/safety constraints |
What counts as "self-improvement" here?
A useful boundary is this: ordinary training updates weights according to a fixed learning rule; the systems in this article are historically interesting because they search over some exposed representation of the solver itself. Sometimes that representation is close to "the model" in the narrow sense; more often it is the software around the model. That difference matters. It is the difference between a learner adapting parameters inside a fixed shell and an outer loop redesigning the shell itself. (Genetic Programming)
This also explains why current agent literature often feels more like software engineering than classical machine learning. Self-Refine explicitly improves outputs through iterative feedback without extra training or RL. Reflexion explicitly says it reinforces language agents "not by updating weights," but through linguistic feedback and episodic memory. Voyager compounds ability through an ever-growing code skill library without parameter fine-tuning. Toolformer is the important counterexample: it does use self-supervised data to train tool use into model behavior, showing that weight-level self-improvement remains possible, but it is not the dominant practical pattern in the current agent stack. (arXiv)
So when modern practitioners say an agent "improves itself," they usually mean one of three things. It may refine its prompt or memory, accumulate reusable tool/skill code, or rewrite the configuration and instruction files that shape future behavior. The commonality is not the substrate but the loop: expose, evaluate, revise. (arXiv)
EURISKO: heuristics made editable
EURISKO is easiest to understand as Lenat's answer to the failure mode of his earlier system, AM. Lenat and Brown later argued that AM had worked in part because mathematical concepts were represented as small Lisp functions, and simple mutations of those functions often yielded neighboring, still-meaningful mathematical concepts. The same approach failed badly for heuristics: when the mutation operators were applied to large heuristic programs, the results were usually useless garbage. Their fix was representational: heuristics had to be decomposed into smaller, semantically meaningful parts. (Paraprogramming Dispatches)
That is the key EURISKO move. In Lenat's 1983 account, EURISKO was written in RLL, a "self-describing and self-modifying representation language." Its explicit ambition was not merely to search a single domain, but to use the same discovery machinery across several domains and to make genuinely new useful discoveries. Most importantly, EURISKO did not just find domain-level concepts and conjectures; Lenat emphasizes that it could find "new heuristics" and also "new heuristics about how to find new heuristics." That is a real metalevel loop, not just brute-force search. (Northwestern CS)
Mechanism. EURISKO treated heuristics as first-class objects and allowed higher-order heuristics to generate, prune, and generalize other heuristics. The system's agenda machinery then selected which proposed changes to work on next. This was not theorem-proving or gradient descent; it was heuristic search over a knowledge base whose own control knowledge was partly editable. (Paraprogramming Dispatches)
What was modified. Not weights, and not yet ordinary source code in the later GP sense, but symbolic control knowledge. Lenat and Brown describe the shift from two-page Lisp heuristic implementations to heuristics represented as collections of roughly twenty or more slots, each containing much smaller units of meaning. Lenat also reports that the slot vocabulary itself expanded over time: eventually generic IF/THEN structure was replaced by specialized slot types, and "for every seven heuristics EURISKO finds, on the average, a new kind of slot is defined." EURISKO therefore modified both content and parts of the representation schema used to express content. (Cantaloupe Image Server)
Feedback signal. EURISKO used internal notions of heuristic performance and "worth," but it also relied on domain outcomes. In the famous Traveller fleet-design work, fleets were generated, fought simulated battles, and battle outcomes were analyzed for design policies and loopholes. Lenat reports that EURISKO had spent about 1300 CPU hours on a Xerox 1100 running this search. The most important detail, though, is not the tournament result but the human supervision around it: Lenat says he culled runs every twelve hours or so, weeded out invalid or undesirable heuristics, and rewarded heuristics he liked. He explicitly estimates the final credit for the win at roughly 60/40 Lenat/EURISKO and says neither alone would have won. (Northwestern CS)
Limiting factor. EURISKO did not become an unbounded self-improver because its success depended on three things that were not self-generated. First, the representation had to be "well matched" to the mutation operators; that was the whole lesson of AM's failure. Second, the knowledge base and slot vocabulary had to be engineered deeply enough for search to land on semantically meaningful variants. Third, the evaluation loop still depended on human judgment and a domain-specific reward structure that could be gamed by loophole discovery. Lenat's later programmatic statement—"slowly hand-code a large, broad knowledge base" before expecting discovery to scale—reads in retrospect as an admission that metalevel search alone does not remove the knowledge bottleneck. (Paraprogramming Dispatches)
EURISKO's deepest legacy is therefore not that it achieved recursive self-improvement in the strong sense. It is that it made the central dependency visible: self-improvement is only as good as the granularity and semantics of the objects being edited. Knowledge Representation and Heuristic Search are not side topics here; they are the topic. (Paraprogramming Dispatches)
Genetic programming: from editable heuristics to editable programs
Genetic programming inherits EURISKO's central abstraction—make the current solver a search object—but discards most of the symbolic metaknowledge. In Koza's formulation, the aim of automatic programming is to tell the computer what needs to be done rather than explicitly programming the steps. Because many target solutions are programs of unknown size and shape, GP represents them as trees of functions and terminals, not as fixed-length strings. That representation shift is historically important: it moves the mutable object from symbolic heuristic frames to executable programs. (Genetic Programming)
Mechanism. GP generates a population of programs, executes them on task-specific fitness cases, and applies reproduction and crossover to the more successful ones. Koza describes the core process plainly: populations of programs are "genetically bred," and the search is "driven by a fitness measure." In GP, self-improvement is thus population-level and evolutionary rather than introspective. The system does not reason about why a modification should help; it samples variants and lets the fitness function decide. (Genetic Programming)
What was modified. The editable object is executable code. Terminals and primitive functions are assembled into trees; crossover swaps subtrees between parents; the best program at termination becomes the result. Later GP work went beyond plain code trees. Koza's work on automatically defined functions (ADFs) and architecture-altering operations let evolution discover reusable subroutines and even parts of program architecture dynamically during the run. In other words, GP was already exploring both code and modularity long before NAS made architecture search famous. (Genetic Programming)
Feedback signal. Fitness is the entire story. Koza says identifying the fitness function is the "critical step" and famously writes that "fitness begets structure." The fitness cases can be as concrete as moving a robot-controlled box closer to a wall or as abstract as solving symbolic tasks, but the logic stays fixed: the external evaluator translates task performance into reproductive pressure. (Genetic Programming)
Limiting factor. GP did not solve the evaluator problem; it centralized it. Humans still had to define the primitive vocabulary, fitness function, termination condition, and often other search parameters. Evaluation could also be expensive. Koza's robot example contrasts GP with reinforcement learning partly on the grounds that RL required many trials over many state-action combinations; GP replaced that burden with large-scale fitness evaluation over candidate programs. On top of that, GP developed a major "bloat" problem: tree size could grow substantially without corresponding fitness gains, consuming resources and distorting the search trajectory. (Genetic Programming)
Historically, GP is where self-improving software becomes easier to industrialize conceptually and harder to romanticize philosophically. It is unmistakably a self-improving procedure, but the improvement is fully bounded by the search space and the externally authored notion of fitness. That is the template AutoML inherits next. (Genetic Programming)
AutoML: narrowing the search space to make improvement practical
AutoML is best understood as a strategic retreat from GP's openness. Instead of evolving arbitrary code, systems such as Auto-WEKA and auto-sklearn search a structured space of machine-learning design choices: which algorithm to use, which preprocessing steps to apply, and how to set their hyperparameters. Auto-WEKA called the combined space "staggeringly large," but the important fact is that it was still a structured space, far narrower and more regular than arbitrary program synthesis. (arXiv)
Mechanism. The dominant AutoML pattern is Bayesian optimization over a hierarchical search space. Auto-sklearn defines the AutoML problem as automatically producing test predictions for a new dataset under a fixed computational budget. It then instantiates that problem with 15 classifiers, 14 feature preprocessing methods, 4 data preprocessing methods, and 110 mostly conditional hyperparameters, while also incorporating past performance on similar datasets and constructing ensembles from evaluated models. This is self-improvement in a very concrete sense: the system repeatedly rewrites its own pipeline design choices in response to validation outcomes. (NeurIPS Papers)
What was modified. The mutable object was no longer code in the general GP sense. It was pipeline scaffolding: algorithm family, data preprocessing, feature preprocessing, and hyperparameters. The candidate model's internal weights still changed during ordinary training, but the self-improvement loop operated at the level of model selection and pipeline composition. That distinction matters. AutoML made improvement more tractable by restricting the editable object to a space with strong prior structure. (NeurIPS Papers)
Feedback signal. AutoML systems optimize validation performance under budget constraints. The auto-sklearn formulation makes the budget explicit—CPU or wall-clock time and memory usage count as part of the problem definition. Feedback is thus a combination of validation loss and resource consumption, not pure task score. This is a much more production-like evaluator than the abstract fitness cases of early GP. (NeurIPS Papers)
Limiting factor. AutoML made search practical, but it did not make evaluation trustworthy by default. Recent work on CASH distinguishes two forms of meta-overfitting: selection-based overfitting, where trying many configurations increases the odds of finding one that looks good on validation by chance, and adaptive overfitting, where Bayesian optimization increasingly tailors the search to quirks of the validation procedure itself. The core limitation is therefore not that AutoML cannot search; it is that the search can outrun the statistical validity of its own evaluator. (ADA Research Group)
AutoML's historical significance is that it made self-improving design loops routine in machine learning. Its philosophical significance is more modest: it achieved that by sharply reducing what counted as "self." The object being improved became the pipeline wrapper, not arbitrary code or open-ended concepts. AutoML and Hyperparameter Optimization are where self-improvement became a mainstream engineering practice. (arXiv)
Neural architecture search: making the network graph itself searchable
Neural architecture search extends the AutoML pattern one level deeper. If preprocessing stacks and hyperparameters are searchable, why not the network topology itself? Zoph and Le's 2016 NAS paper gave the canonical early answer: use an RNN controller to generate candidate architectures, train the controller with reinforcement learning, and reward it with validation accuracy from the child networks it proposes. The architecture becomes just another design object in an outer optimization loop. (arXiv)
Mechanism. NAS uses outer-loop search over architecture descriptions. The search algorithm varies by paper: reinforcement learning in the original NAS, regularized evolution in AmoebaNet, parameter-sharing controllers in ENAS, and continuous relaxation with gradient descent in DARTS. What unifies them is the move from "choose a pipeline" to "choose a graph." Architecture search takes the program-synthesis instinct of GP and the structured-space discipline of AutoML and applies both to neural design. (arXiv)
What was modified. The primary editable object is architecture: cells, operations, connectivity patterns, or the discrete choices encoded in a search space. Some NAS variants also update search-time weights. ENAS explicitly shares parameters across child models inside a larger computational graph, while DARTS relaxes architecture decisions into continuous parameters optimized by gradient descent. NAS therefore sits between "architecture search" and "weight-aware search," but its target remains the network design, not open-ended system redesign. (arXiv)
Feedback signal. As in AutoML, the core evaluator is held-out task performance—validation accuracy, perplexity, or related metrics—sometimes combined with compute constraints. Early NAS made the reward signal expensive because each candidate architecture needed substantial training. One-shot and differentiable methods tried to amortize that cost by sharing weights or replacing discrete search with continuous optimization. (arXiv)
Limiting factor. The first brake was compute. Bender et al. note that early NAS was "incredibly resource-hungry" and cite Zoph et al. as using 450 GPUs for four days on a single experiment. The second brake was epistemic: once one-shot methods reduced search cost, it became clearer that weight sharing, controller design, and benchmark protocol could distort the ranking of architectures. Bender et al. show that neither hypernetworks nor RL controllers are strictly necessary to get strong results in one-shot search, and Li and Talwalkar argue that NAS is largely a specialized hyperparameter-optimization problem where random search is a competitive baseline and reproducibility remains poor. NAS did not abolish human design; it embedded human design in the search space. (Proceedings of Machine Learning Research)
This is the pivotal transition into the modern agent era. NAS normalizes a design philosophy in which a powerful base learner is surrounded by an automatically optimized shell. Modern agents inherit that logic, but move the shell higher up the stack into text and files. Neural Architecture Search and Meta-Learning are the natural companion entries. (arXiv)
Modern AI agents: self-improvement moves into scaffolding
The contemporary agent era does not simply continue NAS. It changes the mutable substrate again. In the most visible agent systems since 2023, the base model is often held fixed while the agent iteratively rewrites prompts, memory buffers, tool descriptions, code libraries, skill manifests, and eventually parts of its own agent codebase. That is a decisive historical shift. Self-improvement becomes less about retraining the model and more about engineering the context, interfaces, and software that shape the model's behavior. (arXiv)
A useful prehistory is tool use. ReAct showed that LLMs could interleave reasoning traces with actions, using tools or environments as part of the problem-solving loop. Toolformer pushed further by training a language model, in a self-supervised way, to decide when to call APIs and how to incorporate the results. These are not yet file-level self-modifiers, but they establish the core modern premise: agent capability depends heavily on the external tools and interface descriptions surrounding the model. (arXiv)
Prompt and memory self-improvement
Self-Refine is the cleanest stripped-down example of test-time self-improvement. A single LLM produces an initial answer, generates feedback on that answer, and then refines it iteratively, explicitly without additional training, supervised data, or reinforcement learning. Reflexion makes the same move in agent form: rather than updating weights, the agent reflects in language on task feedback and stores the reflection in episodic memory for later attempts. (arXiv)
Mechanism. Generate → critique → revise. What was modified. Prompt content and memory text, not model weights. Feedback signal. Free-form self-critique, scalar task outcomes, or environment responses. Limiting factor. These methods often improve behavior locally, but unless the reflections are converted into durable artifacts—code, files, skills, eval rules—the gains remain short-horizon and task-bounded. (arXiv)
Toolformer is the contrasting branch. It teaches the model to use tools through self-supervised weight updates, which means that in the agent era there really is a live lineage of weight-level self-improvement. But the better-documented and more operationally dominant pattern in agents has been the opposite choice: keep model weights fixed and optimize the surrounding scaffold. (arXiv)
Skill accumulation and tool-making
Voyager shows what happens when the result of improvement is made durable. Its key components are an automatic curriculum that maximizes exploration, an ever-growing library of executable code skills, and an iterative prompting mechanism that uses environment feedback, execution errors, and self-verification to improve programs. Crucially, it does all this while interacting with GPT-4 as a black box and "bypassing the need for model parameter fine-tuning." The core idea is straightforward and historically important: agent capability can compound by accumulating reusable code, not just by changing prompts. (arXiv)
LATM and ToolMaker take the same principle from "skills" to "tools." LATM separates a powerful tool-maker model from a cheaper tool-user model and caches newly created tools for future requests. ToolMaker operationalizes that more aggressively: given a scientific paper, its repository, and a task description, it builds a Docker environment and a Python function that downstream agents can call, using a closed-loop self-correction mechanism for debugging and reporting 80% correctness on TM-BENCH. That is not merely tool use; it is automated tool creation as a form of self-extension. (arXiv)
Mechanism. Accumulate reusable code artifacts. What was modified. Skill libraries, tool code, and environment setup. Feedback signal. Environment feedback, execution errors, unit tests, and benchmark correctness. Limiting factor. Tool generation remains sensitive to environment setup, dependency resolution, debugging quality, and benchmark coverage. The agent becomes more capable, but also more exposed to the ordinary brittleness of software systems. (arXiv)
Prompts, tool interfaces, and evaluators as first-class artifacts
Promptbreeder is the clearest modern descendant of both GP and EURISKO. It evolves a population of task prompts according to fitness on a training set, but it also improves the "mutation prompts" that generate prompt mutations. That is genuinely self-referential: the system is not only improving the object-level prompt, but the meta-level operator that proposes future improvements. In historical terms, it is one of the few modern systems that partially recovers EURISKO's "heuristics about finding heuristics" at the prompt layer. (arXiv)
Recent work also makes a subtler claim: the tool interface is itself a bottleneck. A 2026 paper on rewriting tool descriptions argues that agent performance depends heavily on natural-language descriptions and parameter schemas, especially when choosing among many tools, and reports gains on unseen tools and tool sets above 100 candidates. Anthropic's own engineering guidance makes the practical version of the same claim: even small refinements to tool descriptions can dramatically improve held-out performance, including on SWE-bench Verified. The implication is sharp: in modern agents, interface text is not documentation pasted on top of the system; it is part of the optimized system. (arXiv)
This is also where evaluation criteria become part of the self-improvement story. OpenAI's evals guide describes a standard loop: define the task as an eval, run it on test inputs, analyze the results, and iterate on the prompt. The prompt optimizer extends that loop by using datasets, human annotations, written critiques, and grader outputs to improve prompts, while explicitly warning that optimized prompts should still be manually reviewed because they can perform worse on specific inputs. OpenAI's January 2026 guide to testing agent skills turns the implication into engineering policy: skills should become artifacts you can test, score, and improve over time, not just vibes-based prompt bundles. (OpenAI Developers)
There is an important honesty clause here. Evidence for agents autonomously inventing their own evaluators is much weaker than evidence for agents optimizing against human-authored evaluators. The primary sources above consistently assume that humans define the objective, collect the data, and build the graders. What is new is not that agents have escaped external goals. What is new is that evaluators, prompts, and interface descriptions have become editable, versioned software components inside the improvement loop. (OpenAI Developers)
File-based self-modification: the current practical frontier
As of early 2026, the most concrete frontier is file-based agent scaffolding. OpenAI defines a skill as a "versioned bundle of files" plus a SKILL.md manifest and notes compatibility with the open Agent Skills standard. Its Codex guidance says AGENTS.md files are read before work begins, with global defaults, repository-level instructions, nested overrides, and fallback filenames configurable in ~/.codex/config.toml. GitHub Copilot and Anthropic now expose closely related filesystem conventions: Copilot uses SKILL.md directories whose instructions are injected when the skill is chosen, and Anthropic describes Skills as reusable, filesystem-based resources that Claude discovers automatically. Claude Code's overview makes the enabling capability explicit: the agent reads the codebase, edits files, and runs commands. (OpenAI Developers)
This convergence is not cosmetic. It means the mutable substrate is now legible and auditable: prompts, manifests, scripts, tool specs, policy files, and eval harnesses live in ordinary directories. Those artifacts can be diffed, rolled back, unit-tested, benchmarked, and improved by the same software loop that edits application code. In practice, that is the "self" that modern coding agents are most able to rewrite. (OpenAI Developers)
Self-editing coding agents
A theoretical foil belongs here. Schmidhuber's Gödel machine proposed a fully self-referential system that rewrites any part of its own code once it has found a proof that the rewrite is useful. Modern "Gödel machine" papers explicitly position themselves against that ideal: DGM says formal proof of net benefit is impractical and replaces it with empirical validation on coding benchmarks. The names are not accidental; they mark a shift from provable self-improvement to benchmark-tested self-improvement. (arXiv)
The 2025 paper A Self-Improving Coding Agent is one of the clearest claims in this genre. Its abstract says an agent equipped with basic coding tools can "autonomously edit itself" and reports gains from 17% to 53% on a random subset of SWE-Bench Verified, with additional gains on LiveCodeBench and synthetic benchmarks. The important substrate is not the base model weights; it is the agent system—the code that orchestrates model calls and tools. (arXiv)
The Darwin Gödel Machine makes the recursion more explicit. Its abstract says the system iteratively modifies its own code "thereby also improving its ability to modify its own codebase," maintains an archive of generated coding agents, and reports improvements in code editing tools, long-context management, and peer-review mechanisms, alongside a jump from 20.0% to 50.0% on SWE-bench. It also notes sandboxing and human oversight, which is a useful reminder that even the strongest current self-improvement claims are still heavily bounded operationally. (arXiv)
Two follow-ons sharpen both the promise and the critique. Huxley-Gödel Machine argues that high benchmark performance is not the same as high future self-improvement potential, naming the gap the "Metaproductivity-Performance Mismatch." Live-SWE-agent claims to be the first live software agent that continuously evolves its scaffold on the fly during runtime, starting from a basic bash-only scaffold and reporting strong benchmark numbers. Together they push the frontier toward agents that change the code shaping later changes, rather than merely changing outputs within a fixed scaffold. (arXiv)
This is the closest current literature comes to recursive self-improvement in a practical software-engineering sense. But the evidence base is still thin and contested. These are recent arXiv papers, not a long-settled replicated literature; the feedback loops remain benchmark-centered; and HGM's central critique is precisely that benchmark performance may be a poor proxy for future self-improvement capacity. There is no primary-source basis yet for claiming that these systems exhibit unbounded autonomous improvement. What the evidence supports is narrower and more interesting: agents can now rewrite the code and files that shape later agent behavior, and benchmark that loop. (arXiv)
What the lineage actually shows
From EURISKO to modern agents, the main invariant is architectural. Expose a representation, search it, score the variants, retain the winners, repeat. The representations changed—heuristic frames, program trees, pipeline configurations, architecture graphs, then prompts, tool descriptions, skill bundles, and agent codebases—but the outer-loop logic barely did. Self-improvement, historically, is less a singular breakthrough than a recurring pattern for turning parts of a system into searchable objects. (Northwestern CS)
The same four brakes also recur. Representation: AM and EURISKO showed that mutation fails when the representation slices the object at the wrong semantic grain. Evaluation: GP lived and died by fitness design; AutoML by validation fidelity; modern agent stacks by graders and task-specific evals. Cost: NAS made the compute bill impossible to ignore, and tool-making agents rediscover ordinary software-debugging expense. Transfer: HGM's "metaproductivity" argument is the latest form of an old problem—today's score may not predict tomorrow's capacity to keep improving. (Paraprogramming Dispatches)
The current frontier is therefore real, but narrower than the phrase "self-improving AI" suggests. The strongest documented systems today usually do not rewrite frontier-model weights online. They rewrite the software around the model: prompts, memory, tool descriptions, skills, configuration files, and test harnesses; sometimes they rewrite the agent code that decides how to rewrite those artifacts next. That is already powerful enough to matter, precisely because those artifacts are modular, executable, and auditable. It is also why unbounded improvement remains absent: agents improve to the granularity—and inherit the blind spots—of the scaffolding and evaluators we expose to them. (arXiv)
Companion entries
Doug Lenat AM (Automated Mathematician) Cyc Knowledge Representation Heuristic Search Genetic Programming AutoML Bayesian Optimization Neural Architecture Search Prompt Optimization AI Evals Tool Use in LLMs Agent Skills Software Engineering Agents Gödel Machines Goodhart's Law