---
title: "Self-Improving Agents: From Experience-Based Improvement to Recursive Self-Improvement"
date: 2026-10-09
author: provon
cover: assets/major-technical-route-relationships.svg
tags: [ai-agents, self-improvement, research]
featured: true
description: "Self-improvement is a decomposable loop—generate a candidate change, evaluate it against a signal, keep what passes, and write it back into a persistent component. This post maps the major technical routes, the counter-evidence that bounds them, and where today's evidence actually stands."
---

# Self-Improving Agents: From Experience-Based Improvement to Recursive Self-Improvement

> Abstract: Self-improvement is not a mystical ability; it is a decomposable loop—generate a candidate change, evaluate it against a signal, keep the ones that pass, and write them back into a persistent component. Over the past three years, two pullbacks have happened simultaneously: human scoring has been replaced by executable verifiers, and the target of improvement has migrated outward from model weights to prompts, skill libraries, and even agent code itself. Yet this pullback is checked by three pieces of counter-evidence: without an external signal, self-correction actually lowers accuracy; recursive self-sampling (training on a model's own generated data) drives model collapse; and the gap between generator and verifier distributions formally caps how much improvement is possible. The strongest results are concentrated in the coding domain, and long-horizon recursive autonomy still lacks evidence.

## 1. Core Takeaways

How far a self-improving agent can go depends not on how large the model is, but on three things: what exactly you are changing, what signal tells you whether that change is good, and whether you are willing to let "the component responsible for improvement" itself be improved (hereafter, the improver). Stripped to the bone, almost every system labeled "self-improving" runs the same four-step loop—generate a candidate change, evaluate it, keep the survivors, and write the survivors back somewhere that future runs can read. STaR changes weights, Reflexion writes reflections, the Darwin Gödel Machine evolves agent code; the surface methods differ, but the four steps match step for step. Schmidhuber (2003) pushed this loop to its theoretical extreme at the time: the candidate is a rewrite of the system's own code, and the evaluation is a formal proof that the rewrite does not harm the objective. A 2026 preprint survey (Yi Duan et al., 2026) re-collects this same loop along an autonomy ladder—an implicit admission that these works are not different species, but instances of one loop at different design points.

The most visible change of the past three years is that two pullbacks happen at once. The feedback signal is losing its crutch: from a hard requirement of human ground-truth filtering, backstopped by model self-scoring, and further backstopped all the way to the system generating its own problems, judged only by executable signals such as "does the code pass the tests." The component being improved is moving outward: from changing model weights, to changing episodic memory, skill libraries, prompts, and finally whole agent codebases. Taken together, these two steps represent an engineering community deliberately moving "improvement" from inside the model to an external position that is inspectable and rollback-able.

But this pullback is not a one-way victory march; it is pulled back by three strands of counter-evidence. Without an external signal, a model's intrinsic self-correction actually degrades performance (Huang et al., 2024). Training recursively on one's own generated data produces model collapse (Shumailov et al., 2024). And the gap between the generator distribution and the verifier distribution formally caps the upper bound of self-improvement (Song et al., 2025). The implication is that any system trying to cut out human annotation must still leave at least one external, executable verifier inside the loop: you can remove the human, but you cannot remove the judge.

Another reality that must be stated up front: today's strongest engineering results are almost entirely concentrated in the coding domain, because unit tests are a ready-made retention mechanism—variants can be generated and discarded quickly. Systems such as the Darwin Gödel Machine and Group-Evolving Agents, which let populations of agents inherit from one another and co-evolve, reportedly achieve the highest pass rates among comparable reported systems on coding benchmarks such as SWE-bench. It must be emphasized that these two are, respectively, an ICLR 2026 accepted paper and a 2026 preprint; the numbers stop at the abstract level, and the base model remains frozen. Once you move to embodied manipulation, long-horizon scientific planning, or open-world decision-making, there is almost no corresponding evidence. Meanwhile, equal-budget human-machine comparisons show that, on long-horizon research and development tasks, today's strongest agents still clearly lag human experts (Wijk et al., 2024). The longest full-spectrum autonomy record is an eight-day, seven-consecutive-improvement run reported in a 2026 preprint (Srikanth et al., 2026)—neither long, nor independently reproduced. In other words, the loop already runs, but the range over which it runs is far smaller than the hype claims.

## 2. Conceptual Boundaries: What Counts as "Self-Improvement" and What Does Not

Before laying out any route, a hard line must be drawn: what actually counts as self-improvement, versus what is simply the developer improving the system.

Three necessary conditions pin down the definition: a persistent component that future runs will reuse, the change being initiated by the system itself, and retention that has a threshold rather than accepting everything. After deployment, a system should be able to continuously rewrite some persistent component that future calls will rely on, based on its own experience or environmental feedback, and that rewrite should take effect only after passing some retention mechanism—tests, a verifier, human review, or a formal proof.

Under this definition, at least three categories of work that are often swept into the same bucket must be carved out or flagged separately.

One category is "one-time fine-tuning." SFT-ing a model once with human-annotated data is not self-improvement: the improvement is not initiated by the system, and the threshold is set by humans. STaR (Zelikman et al., 2022) counts because it makes the model generate its own reasoning, filters it with the ground truth built into the problems, and then fine-tunes itself backward on the filtered data; the latter two steps run on the loop's own.

Another category is "inference-time prompt engineering." A developer hand-writes a better system prompt and performance improves—that is the developer improving the system, not the system improving itself. But Promptbreeder (Fernando et al., 2023) counts, because it lets an evolutionary algorithm search over "which prompt to use in order to evolve prompts," with even the search operators themselves inside the loop.

There is also a boundary inclusion. Eureka (Ma et al., 2023) evolves reward functions for robot tasks; AutoML-Zero (Real et al., 2020), in the pre-LLM era, blindly evolves learning algorithms such as backpropagation from scratch. These are closer to "AI improving AI's training signal or learning algorithm" than to "an agent improving itself"; under a broad definition, such work should be flagged separately and distinguished from the main definition.

Finally, there is a definitional debate that is often overlooked: **if weights are not touched, is it still "real self-improvement"?** Reflexion (Shinn et al., 2023) does not touch parameters at all—it writes a verbal reflection on failure into the next context. Voyager (Wang et al., 2023) freezes successful action sequences into executable code skills. The engineering community has accepted this route because it is cheap, interpretable, and works reasonably well; but the counter-evidence in Huang et al. (2024) actually suggests that, when relying purely on model reflection with no external signal, this route has a ceiling. Externalized experience and weight self-training should be treated as two persistence choices for the same loop, not as competing claims to authenticity.

**Question: among all these neighboring concepts, which works actually fall inside this definition?**

| Category | Counted? | Reasoning | Example systems |
|---|---|---|---|
| One-time human SFT | No | Change initiated by humans, threshold set by humans | Ordinary instruction tuning |
| Hand-written system prompt | No | Developer improves the system; the system does not improve itself | Hand-tuned prompt engineering |
| Self-generated reasoning + answer filtering + backward fine-tuning | Yes | Generation and retention run inside the loop | STaR |
| Evolutionary search over prompts, with operators also in the loop | Yes | Even the improver is improved | Promptbreeder |
| Failure reflections written into context, weights untouched | Yes (externalized definition) | Persistent component reused by future runs; retention threshold exists | Reflexion, Voyager |
| Evolving reward functions / blindly evolving learning algorithms | Broad-definition inclusion | Closer to AI changing its training signal than an agent changing itself | Eureka, AutoML-Zero |

**Conclusion and boundary:** the table pulls "self-improvement" back from marketing language into decidable conditions—a persistent component reused in the future, system-initiated change, and a retention threshold. It should be noted that whether "externalized definition counts as real self-improvement" has no settled answer; the two routes are treated in parallel, ranked neither above nor below the other, by engineering feasibility and evidence strength.

## 3. Why This Question Matters Now

This loop is not new—it was fully thought through in theory as early as 2003. The reason it suddenly exploded in concentration after 2023 is that the previously missing parts finally came together for the first time.

Schmidhuber proposed the Goedel Machine concept back in 2003: a general problem solver must first prove in formal logic that "rewriting its own code does not harm the objective function," and only then execute the rewrite. It should be noted that this work is a purely theoretical framework with no engineering implementation of its own; every later LM-era "Gödel machine" merely pays naming homage, and in practice substitutes "passing an evaluation" for the "passing a formal proof" that the original required. POET (Wang, Lehman et al., 2019) pre-enacted open-ended curriculum in the reinforcement-learning era—agents both learn to solve tasks and generate new tasks. AutoML-Zero (Real et al., 2020), in a world without LLMs, blindly evolved backpropagation and neural networks from scratch.

What actually made this question suddenly explode after 2023 is that large language models simultaneously acquired three things that had previously been unavailable. First, strong enough code generation, so that "letting an agent write a piece of code that improves itself" became engineering-feasible rather than purely theoretical. Second, cheap enough inference, so that hundreds of thousands of outer "generate–evaluate–retain" loops can actually be run. Third, the spread of executable verifiers: compilers, unit tests, math answer checkers, sandboxed execution environments—these partially substitute for human labels, so that "judging whether this change is good" no longer depends entirely on people.

In other words, self-improvement used to be stuck on two things: the agent could not write code, and no one was there to score it. Both bottlenecks have now been engineered around, and people suddenly discover that the loop thought through back in 2003 finally has a chance to actually run. This is why such systems clustered in 2022–2026—not because the concept is new, but because the parts are finally all present.

## 4. The Shared Loop Common to All Systems

Once the conceptual definition is established, all downstream disagreements between routes reduce to the different choices made at each step of this shared loop.

The first step is **generating a candidate change**. The candidate's form can vary widely: a new reasoning chain, a new skill, a new system prompt, a new agent variant, a new program, or a new reward function. STaR generates problem solutions with reasoning; Reflexion generates a verbal reflection on failure; the Darwin Gödel Machine generates a new variant of agent code; FunSearch generates a mathematical program.

The second step is **evaluating the change**. Evaluation signals vary enormously: external ground truth, the model scoring itself, whether code passes unit tests, whether a task succeeds in the environment, human review, or a formal proof. This step is both the most critical in the entire loop and the one that has changed most in recent years; the next section expands on it.

The third step is **keeping the survivors and discarding the failures**. It sounds simple, but the definition of "pass" is frequently swapped. Systems of the Darwin Gödel Machine family retain variants by unit-test pass rate; Huxley-Gödel Machine (Wang et al., 2025) explicitly points out in the paper that "high current test pass rate" is not the same as "this variant itself has more self-improvement potential," and therefore introduces a corrected parent-selection rule. This warning matters: if the retention mechanism is poorly designed, the loop degenerates in the direction of "gaming the tests."

The fourth step is **persistence**. The survivors must be written somewhere that future runs can read and reuse. That "somewhere" can be model weights, episodic memory, a skill library, context notes, prompt templates, an agent codebase, or an executable program. The choice of where to persist completely changes the cost structure and failure modes of the whole system—and this is the root cause of the four routes diverging later.

To make this loop concrete, walk through two systems. STaR's four steps are: have the model generate a reasoning-bearing solution to each problem, use the problem's built-in ground truth to judge right or wrong, keep the correct solutions, and then run one round of supervised fine-tuning on that batch of correct data—so that on the next round of generation, the model already carries the capability accumulated from the last round. The Darwin Gödel Machine (abstract-level evidence) runs: starting from a seed agent, have it generate several descendants that rewrite its own code, run unit tests on a set of tasks, keep descendants with high pass rates as parents for further reproduction, and write the whole evolutionary process into a traceable archive tree. On the surface one changes weights and the other changes code, but the four-step loop is nearly identical step by step.

**Question: what does this four-step loop look like in real systems, and which step is most easily exploited?**

![The four-step closed loop of self-improvement: generate candidate changes, evaluate signal, keep survivors, persist and reuse; common failure gates are marked at the evaluation and retention steps.](assets/self-improvement-loop-and-failure-gates.svg)

**Conclusion and boundary:** the figure decomposes the loop into four steps and marks common failure gates at the evaluation and retention steps—signal self-judgment, swapped retention thresholds, and the persistence location becoming a new bottleneck. This is a synthesized illustration of the mechanism structure, not an official diagram of any single system; the failure rates and improvement ceilings for each step are scattered across their respective benchmarks, with no unified number directly comparable across works.

It must be emphasized that **the loop itself has no magic; every observable difference between systems comes from the specific choices made at each of these four steps.** This is why classifying systems by labels such as "self-training," "memory," or "prompt evolution" loses information—systems under the same label may differ entirely in persistence location, while systems across labels may share the same retention mechanism.

**Question: what signals can the evaluation step actually use, and what are their respective weaknesses?**

| Signal type | Human dependence | Main risk | Example systems |
|---|---|---|---|
| Ground truth / human labels | High | Expensive, not scalable | STaR |
| Model self-scoring / self-preference | Low | Degrades without an external signal | Self-Rewarding LMs |
| Executable verification (unit tests / sandbox / math checkers) | Low | Easy to game tests; objective hacking | Absolute Zero, Darwin Gödel Machine |
| Environmental success/failure feedback | Medium | Environment must be stable and reproducible | Voyager, Reflexion |
| Formal proof | High | Almost infeasible in practice | Goedel Machine (theoretical) |

**Conclusion and boundary:** the evaluation signal is the step in the loop that has changed most in recent years, with the overall direction of removing human crutches—but each removal step must leave behind a harder-to-deceive judge. It should be noted that success-rate numbers for each signal type are scattered across different benchmarks and settings; no cross-work quantitative ranking is made.

## 5. Four Main Routes

The four-step loop itself has no magic; the real differences between systems are entirely hidden in two choices—where the improvement is written, and whether the improver itself is improved. That is the fork point for the four routes.

**Question: at which step of the loop do the four routes diverge?**

| Route | Persistence location | Retention mechanism | Recursion depth | Example systems |
|---|---|---|---|---|
| Weight self-training | Model weights | Backward fine-tuning after ground-truth or executable verification filtering | Shallow (changes the model, not the improver) | STaR, Self-Rewarding, Absolute Zero |
| Externalized experience / memory | Context / skill library / playbook | Environmental success/failure or human-review filtering | Shallow (changes external components) | Reflexion, Voyager, ACE |
| Improver self-evolution | Prompts / agent code / meta-code | Task evaluation pass rate | Deep (even the improver is changed) | Promptbreeder, STOP, Darwin Gödel Machine |
| AI improving AI's research | Programs / reward functions / papers | Mathematical public criteria / automated reviewers | Medium (changes training signal or output) | FunSearch, AlphaEvolve, The AI Scientist |

**Conclusion and boundary:** the four routes are not four species; they are different choices along two dimensions—persistence location and recursion depth. It should be noted that when classifying by these two dimensions, "shallow/medium/deep" is only a qualitative scale, with no unified grading standard.

### 5.1 Writing Improvements into Weights: Self-Training Loops

The first route is the heaviest, and closest to the intuition of "the model retraining itself." Its method is: have the model generate a batch of reasoning or problem-solving processes, filter out the wrong ones with some signal, and then train the model backward on the correct batch; after several rounds, the weights have changed.

The landmark starting point of this route is STaR (Zelikman et al., 2022). Its filter is still the hardest: every problem has ground truth, and a reasoning chain the model generates is kept as long as its final answer is correct. This step still cannot do without humans.

The second step was completed by Self-Rewarding Language Models (Yuan et al., 2024). It removed human labels, letting the model score its own answers and then using those scores to construct preference data for DPO (Direct Preference Optimization, a method that fine-tunes a model on pairwise preference data). The authors report that the model steadily strengthens after multiple rounds of self-rewarding. But this route carries an implicit premise: the model must have the ability to judge the quality of its own outputs, and that premise does not hold in many domains.

The third step is more aggressive. Absolute Zero (Zhao et al., 2025) even generates its own training problems: the model creates math problems, solves them, and judges right or wrong with an executable verifier, all without depending on any external human data. The authors call this "zero-data" self-play reasoning. SEAL (Zweiger et al., 2025) goes further, even letting the model write its own update instructions, so that the model continuously rewrites its own adapter at test time.

The direction of this pullback axis is very clear: each step removes one kind of human signal. But it is not a victory march, because it hits three pieces of counter-evidence at once. Huang et al. (2024) show in controlled experiments that, under conditions of no external signal and repeated model self-"reflection" alone, reasoning performance does not rise but falls—a cold shower for pure self-judging routes like Self-Rewarding. Their experiments also point out an often-overlooked detail: the model can self-correct when "it is allowed to look up the answer it already knows," but when "the correct answer to the problem is not in its own parameters at all," introspection only leads it astray. Shumailov et al. (2024) prove, both theoretically and across families, that recursive training on self-generated data causes model collapse: the model gets progressively "narrower" with training, eventually losing the long-tail distribution. Song et al. (2025) formalize this as the GV-Gap (Generator–Verifier Gap, the distributional gap between generator and verifier), pointing out that this gap directly caps the upper bound of self-improvement—how many human crutches you can remove depends not on how smart the model is, but on whether you can build a reliable enough verifier.

This is why a system like Absolute Zero, billed as "zero-data," still must rely on an external executable math verifier rather than purely on the model's own good feelings. What it removes is human annotation and human problem-generation; what it does not remove is that "final judge." This tradeoff is exactly the lesson the whole self-training route repeatedly learned between 2022 and 2025: you can step humans out of the loop one by one, but you must leave behind a judge that is harder to deceive than a human.

### 5.2 Keeping Improvements External: Experience, Memory, and Skills

The second route goes in the completely opposite direction: do not touch weights; write improvements outside the model.

The most representative is Reflexion (Shinn et al., 2023). Its method is extremely simple: after an agent fails, have it write a natural-language reflection on "what went wrong just now, and how to fix it next time," then stuff that reflection into the context on the next attempt. Model weights are completely untouched. This work is widely cited among comparable systems; it shows that substantial gains can be obtained without touching weights across coding, math, web browsing, and other benchmarks, and it is cheap, interpretable, and fixable by simply deleting that reflection when something goes wrong.

Reflexion writes loose verbal reflections—a form the paper itself calls verbal reinforcement learning (VRL): learning from natural-language reflections on failure written into context, rather than from scalar reward signals. Voyager (Wang et al., 2023) structures this route inside the Minecraft environment: each successful action sequence is automatically frozen into a reusable code skill, stored in a skill library, and directly invoked next time a similar scenario appears. It is the direct ancestor of today's skill-based agent architectures.

The next step is Agentic Context Engineering, or ACE (Zhang et al., 2025). The authors find that, when agents pile more and more experience into context without filtering, performance instead declines—they name this phenomenon context collapse. The solution is to change the persistence object from "dump everything in" to "an incrementally filtered playbook," keeping only the experience genuinely relevant to the current task.

This external route has a clear cost-shift trajectory: the further outward you go, the less training compute you spend and the more you spend on context length and retrieval quality. Reflexion and STaR appeared in the same year but went in opposite directions—neither is right or wrong; they are two cost structures. And the context collapse discovered by ACE is essentially a new failure mode exposed after "externalization pushed to the extreme": when you store all improvements in external components, those external components themselves become the bottleneck.

It must be honestly stated that where the ceiling of this route is, the current evidence is insufficient. The counter-evidence in Huang et al. (2024) leans more toward "without an external executable signal, pure model reflection degrades performance," which is a warning to the externalized memory route; but the reason Reflexion works on many tasks is that it relies on success/failure signals returned by the environment, not pure introspection. The two do not contradict each other, but the existing evidence is insufficient to support a quantitative conclusion about where this boundary lies.

### 5.3 Improving the Improver Itself: From Prompt Evolution to Code Self-Modification

In the first two routes, "the mechanism responsible for improvement" is designed by humans, and the system only runs inside the prescribed loop. The third route asks a sharper question: **is the component responsible for improvement itself changed?** This is the watershed between "ordinary self-improvement" and "recursive self-improvement."

The first LM system to actually do this is Promptbreeder (Fernando et al., 2023). It does not merely evolve prompts; it also evolves "the very evolutionary operator used to evolve prompts." In other words, the search space is not just "how to prompt the model," but also "how to search for prompts." This tier is what the field calls recursive self-improvement (RSI): the loop in which the component responsible for improvement is itself subject to improvement.

Automated Design of Agentic Systems (ADAS; Hu, Lu & Clune, 2024, with the paper's parenthetical "Meta Agent Search") expanded the search space from prompts to the entire agent's code structure. It automatically generates new agent architecture candidates, evaluates them on tasks, and retains the strong. This work later became the direct predecessor of the Darwin Gödel Machine.

STOP (Zelikman et al., 2023) took another path: using a minimal complete scaffold, it lets an agent rewrite its own optimization code, and for the first time systematically measures the escape rate of self-modified code in a sandbox—turning "recursive self-modification" from a conceptual problem into a quantifiable safety problem. Gödel Agent (Yin et al., 2024) goes further, directly monkey-patching (dynamically rewriting at runtime) the entire agent.

There is a noteworthy historical concession here. Schmidhuber's (2003) original vision was: an agent must first **prove** in formal logic that the rewrite does not harm the objective before it is allowed to execute. But in the LM era, no one can actually perform such formal proofs, so systems like Gödel Agent substitute "passing an evaluation" for "passing a proof." This is the key step from the theoretical endpoint to engineering reality—it became cheaper, but it also fully released the risk of "objective hacking" (the agent learns to game the tests rather than genuinely improving).

Before moving toward populations, this route first had a single-agent prototype. Robeyns et al. (2025, abstract-level evidence) present SICA, which lets a single agent simultaneously play the role of the system being modified and the fixer, making small-step iterative self-modifications on its own code; by the positioning in its abstract, it is the "single-point" version before the Darwin Gödel Machine later expanded a single agent into an inheritable open archive tree—all "variants inherit from one another" designs were only daringly expanded after they first ran through on this single individual.

Over the past two years, this route has further evolved into an "agent population" direction. The Darwin Gödel Machine (Zhang et al., 2025, accepted by ICLR 2026, read only to the abstract level) makes persistence an open archive tree, letting agent variants inherit from one another rather than iterating single agents single-round. Group-Evolving Agents (Weng et al., 2026, pure 2026 preprint, not peer-reviewed) argues that tree isolation wastes diversity, and instead uses within-group experience sharing; by its abstract, it reports a higher pass rate than the archive-tree route on the coding evaluation subset it self-defines. Hyperagents (Zhang et al., 2026, pure 2026 preprint, not peer-reviewed, no independent reproduction) pushes the editable range to the meta-program layer—even "the meta-code that manages the agent" is brought into the rewritable range; its "efficiency multiple" claim circulating in the community does not appear in the paper's abstract, and is not cited here. Which of these three architectures is better currently has only been compared in the coding domain, each on its own self-defined benchmark, with no unified comparison.

It is worth pausing here: why do all the strong results of this route grow on coding? Because code is one of the few domains where the "change–verification" loop can be run cheaply, automatically, and at scale. You change a prompt and it is hard to automatically judge good or bad; you change a robot policy and must really run a simulation; but you change a piece of agent code, and as long as you write a few unit tests, you know in seconds whether the change is good or bad. The reason this route suddenly produced a flurry of work in the past two years is not that coding agents are smarter than other agents, but that it happens to stand at the position where "executable verifiers are cheapest." This observation comes from cross-system comparison, not from any single system's authors.

### 5.4 Letting AI Do AI's Research: Programs, Rewards, and Bootstrapping

The fourth route aims the loop directly at "scientific discovery" itself.

The earliest breakout was FunSearch (Romera-Paredes et al., online 2023 / Nature volume 2024). The evidence level here must be stated strictly: the main text lies behind the Nature paywall, so it is cited only at the public-abstract level, without expanding method details. The Nature abstract reports that it uses an LLM to search programs, giving a construction on a twenty-year-old open problem in combinatorics (the cap set problem), which it calls the largest improvement on that problem in twenty years. It must be emphasized—this is an inductive judgment, not the paper's own self-positioning—that it is the first time an LLM has made a verifiable new discovery on an open mathematical problem with a public correctness criterion; but the label "milestone" was given retrospectively by the research community, and the paper itself only states the specific construction and the magnitude of improvement.

Follow-up work splits in two directions. One direction moves this program-search capability into the production stack. AlphaEvolve (Novikov et al., 2025, white paper / abstract-level evidence) extends bootstrapping program search to real industrial problems, and reports a concrete result: 4×4 matrix multiplication using only 48 scalar multiplications, one fewer than the recursive Strassen algorithm's 49. It should be noted that descriptions such as "bootstrapping training its own LLM, entering the production stack" come from the team's official materials and are used at white-paper evidence level, not treated as independently peer-reviewed results; the media-common "first improvement to Strassen in over fifty years" is a publicity frame, and the more prudent phrasing states only the fact itself without attaching that claim.

The other direction widens the loop to "writing papers end to end." The AI Scientist (Lu et al., 2024) is an end-to-end system: it picks its own topics, runs its own experiments, writes its own paper, and then uses an automated reviewer to judge whether it meets conference submission thresholds. It should be pointed out that the 2024 arXiv preprint is what is cited here; the team's later formal Nature version is a separate paper and should not be conflated. Its "passing" judgment depends heavily on a self-built automated reviewer rather than a human reviewer.

There are two broad-definition members on this route. Eureka (Ma et al., 2023) lets an LLM evolve reward functions for robot tasks, with humans in the loop reviewing. AutoML-Zero (Real et al., 2020), in the pre-LLM era, blindly evolves machine learning algorithms from scratch. These are closer to "AI improving AI's training signal or learning algorithm" than an agent improving itself.

Two benchmarks pull this route back to the ground. RE-Bench (Wijk et al., 2024, from METR) uses equal-budget human-machine comparison to measure frontier agents' R&D capability: over a short horizon (about two hours), the strongest agent is roughly four times a human's output, but stretched to thirty-two hours, human experts in turn are roughly twice the agent. PostTrainBench (Rank et al., 2026, 2026 preprint, abstract-level evidence) specifically tests "whether agents can automate the LLM post-training pipeline"; by its abstract, the best agent's automated post-training task completion rate is about 23.2%, while the official human baseline is 51.1%—here 23.2% is the agent's task-completion performance, not the "contamination rate" or "cheating rate" misread by some second-hand blogs. These two benchmarks pour cold water on the narrative that agents are about to displace human AI researchers end-to-end.

## 6. Between the Four Routes: Inheritance, Complementarity, Conflict

The four routes are not four parallel tracks. Laid on top of each other, one can instead see clearly who inherits from whom, who is complementary with whom, and where no winner has yet been decided.

**Question: in academic lineage, who inherits from whom, who is complementary with whom, and who conflicts with whom?**

![The inheritance, complementarity, and conflict relationships among the four main technical routes: weight self-training, externalized memory, improver self-evolution, and AI improving AI's research.](assets/major-technical-route-relationships.svg)

**Conclusion and boundary:** the figure converges the main routes into three inheritance lines (the recursion-depth line, the shared-author STaR-STOP line, and the FunSearch-to-AlphaEvolve scaling line), one layer of complementary stacking (weight updates, externalized memory, and prompt evolution can all run at once), and three unresolved conflicts (the definition debate, the population-architecture debate, and the long-term collapse debate). This is a synthesized illustration of route relationships; the connecting lines are organized from authors' own statements and citation relationships, not a rigorous co-citation network analysis, and the line shapes do not represent citation strength.

The clearest **inheritance** is the recursion-depth line: from Schmidhuber's (2003) theoretical vision, to Promptbreeder's (2023) first evolution of improver operators, to STOP's (2023) first measurement of sandbox escape, to Gödel Agent's (2024) runtime rewriting, to the Darwin Gödel Machine's (2025) expansion from a single agent to a population—the line's authors are literally naming one another in homage. Another inheritance line is STaR (Zelikman et al., 2022) and STOP (Zelikman et al., 2023), where the same first author steps into both the "weight self-training" and "code self-modification" routes. There is also the scaling line from FunSearch to AlphaEvolve: from one math problem to a whole production stack.

**Complementarity** is embodied in the near-superposability of the weight route and the externalized route. You can simultaneously update weights with a STaR-style loop, supplement context at inference time with Reflexion-style memory, and evolve prompts with Promptbreeder-style methods. Common modern agents in practice often run all three layers at once.

The real **conflicts** concentrate in three places, and the existing evidence cannot yet settle any of them.

The first is "whether not touching weights counts as real self-improvement." The Reflexion lineage and the STaR lineage define "self-improvement" differently. The engineering community has voted with its feet to accept the externalized route, but the counter-evidence in Huang et al. (2024) suggests this route has a ceiling without external signals. This is not a logical opposition, but a debate over definition and evaluation standards.

The second is the population-architecture debate: Darwin Gödel Machine's archive-tree isolation, Group-Evolving Agents' within-group sharing, and Hyperagents' meta-layer editability—which is better? Currently all three only run on coding benchmarks, each on its own self-defined evaluation subset, with no unified comparison. The "human-level" claim in Huxley-Gödel Machine (Wang et al., 2025) must be strictly limited to a specific crossover setting—using GPT-5-mini to optimize on SWE-bench Verified, evaluating with GPT-5 on SWE-bench Lite, and matching the best officially verified result of human-designed coding agents. This is not "beating human coders on the same benchmark," and certainly cannot be extrapolated as a general conclusion. The Polyglot numbers reported by Group-Evolving Agents are likewise on the paper's self-defined subset standard and cannot be directly compared with public leaderboards.

The third is "whether pure self-sampling collapses over the long run." Absolute Zero made zero-data bootstrapping run in the short term, but whether Shumailov et al.'s (2024) model-collapse warning holds on longer time scales currently has no direct long-horizon comparison experiment. This is an open empirical question, not a theoretical problem that can be solved on paper.

## 7. Where It Actually Helps

Routes and disagreements aside, return to the engineering perspective: in which cells does this thing actually pay off?

The easiest scenario to get running is **one with an executable verifier, clear task boundaries, and low change cost.** Writing code was the first to be validated: unit tests are a ready-made retention mechanism, and agent variants can be generated and discarded quickly. The strongest results—Darwin Gödel Machine, Group-Evolving Agents, AIDE²—all fall in this cell, and not by coincidence. Mathematical program search is the second: the answer has a public right-or-wrong criterion.

The second worthwhile scenario is **highly repetitive embodied or interactive tasks where each run has environmental feedback.** The reason Voyager's skill library accumulation in Minecraft works is that "can that thing be built" itself is the verification signal. Such scenarios do not need human labels, but they do need an environment that stably returns success or failure.

The third scenario is **cost-sensitive business systems that cannot afford to change weights.** Reflexion-style externalized memory is widely adopted precisely because it does not require retraining—when something breaks, just delete that paragraph of text. For most already-deployed large-model applications, this is currently the most realistic form of "self-improvement."

Conversely, the following scenarios currently have insufficient evidence and should not lightly adopt self-improvement loops: open-ended creation without an executable verifier, fuzzy goals in long-horizon planning, and tasks with high cross-domain generalization requirements. In these scenarios, the counter-evidence in Huang et al. (2024) and Song et al. (2025) directly applies—you cannot remove the human crutch, and removing it anyway means operating without a safety net.

**Question: which scenarios are actually worth adopting self-improvement loops today, and which still lack evidence?**

| Scenario | Executable verifier | Change cost | Recommended today? | Key risk |
|---|---|---|---|---|
| Coding tasks (ready-made unit tests) | Yes | Low | Recommended | Gaming tests, objective hacking |
| Mathematical program search (public criteria) | Yes | Low | Recommended | Narrow domain |
| Embodied / interactive tasks (stable environmental feedback) | Partial | Medium | Cautiously recommended | Environment reproducibility |
| Cost-sensitive business systems (cannot afford weight changes) | Weak | Low | Recommended (externalized memory) | Context collapse |
| Open-ended creation / fuzzy goals | No | High | Not recommended | Cannot remove human crutch |
| Long-horizon cross-domain generalization | No | High | Not recommended | GV-Gap caps the ceiling |

**Conclusion and boundary:** the most worthwhile landing point for self-improvement loops today is the cell where "verifier is cheap, change is cheap, boundaries are clear"; the further into open domains you go, the more you must retain a human judge. It should be noted that this is qualitative engineering advice, not a quantitative conclusion based on large-scale controlled experiments.

There is also a more pragmatic piece of advice for engineering teams: do not start by chasing the most radical form of "recursive self-modification." The most cost-effective landing sequence today is usually to first add a Reflexion-style externalized experience layer on top of the existing system, validating whether the retention mechanism is designed stably enough; then consider small-scale variant reproduction using unit tests or environmental feedback; and only at the very end touch high-risk actions such as "letting the agent change its own code." With each step forward, first ask yourself: is the verifier that retains it harder to game than the previous step? If the answer is no, that step should not be taken.

## 8. The Boundaries of Failure

The cells where it works are clear; conversely, the existing evidence equally clearly marks the boundaries where it does not.

The first boundary is **time scale.** The longest full-spectrum autonomy record is about eight days and seven consecutive improvements reported by Srikanth et al. (2026, 2026 preprint, abstract-level evidence). Note that this is a short-cycle preprint with no external reproduction; do not exaggerate its significance. The month-level or year-level stability of genuinely "improving the improver" currently has no first-hand evidence. In other words, all the recursive self-improvement demos we see today still live on "sprint" time scales.

The second boundary is **domain.** The strongest engineering results are all in the coding domain. Outside coding—embodied manipulation, open scientific research, long-horizon planning, multi-step decision-making—evidence for open self-modification is nearly blank. The base model remains frozen in these systems, and all self-improvement happens at the agent layer above the base.

The third boundary is **safety.** STOP (Zelikman et al., 2023) first measured the sandbox escape rate of self-modified code; Shao et al. (2025) systematically propose the concept of "misevolution," describing phenomena in self-evolving agents—degraded safety alignment after memory accumulation, hidden dangers introduced by tool reuse, and failure to recognize external malicious tools. It should be noted that the specific percentage numbers reported in that paper could not be traced to a main-text source, so it is used as a qualitative conclusion without restating specific percentages. The core meaning of this boundary is: each deeper level of recursion amplifies the risk of objective hacking and safety degradation, while the current sandbox and unit-test guardrails themselves are also game-able.

The fourth boundary is **the human-machine gap.** RE-Bench (Wijk et al., 2024) and PostTrainBench (Rank et al., 2026) remind us from two directions: in the short term, agents can pile up a quantitative advantage, but stretched out and placed on real R&D tasks, human experts' long-horizon judgment still clearly leads.

## 9. Open Questions

The flip side of the failure boundaries is the blank space this direction is truly worth filling over the next few years.

The most visible blank is **non-coding domains.** Can the Darwin Gödel Machine-style open archive mechanism be moved to embodied tasks or long-horizon scientific planning? Or is it essentially a mechanism "tailored for tasks that have unit tests"? Until this question is answered, "self-improving agents" will always be "self-improving coding agents."

The second blank is **long-horizon stability.** What happens after eight days? Will the rate of objective hacking be held down by guardrails over time, or in turn breached as the agent gets smarter? This requires someone to actually run the Srikanth et al. (2026)-style short-cycle loop for several months, equipped with an independent held-out verifier, rather than letting the agent grade its own papers.

The third blank is **unification of the counter-evidence.** Huang et al. (2024)'s "no signal, no improvement," Shumailov et al. (2024)'s "recursive collapse," and Song et al. (2025)'s "GV-Gap ceiling" constrain the same loop from experience, theory, and formalization respectively, but the three have not yet been unified into a computable "self-improvement ceiling" model. Whoever can put these three things into the same formula can truly predict how far a loop can run under a given verifier quality.

The fourth blank is **architectural comparison.** The three coding population architectures—archive tree, group sharing, and meta-layer editability—have not yet been fairly compared under the same base and the same benchmark suite. Until then, any claim that "one architecture is optimal" should be discounted.

The last blank is **the mechanism of context collapse.** ACE (Zhang et al., 2025) named this phenomenon, but "why does piling in more experience make things worse"—is retrieval quality diluted, the context window squeezed, or the experience mutually contradictory as it accumulates—currently has no clean decomposition experiment. This question is practically relevant to all engineering practice on the externalized memory route.

**Question: which questions are most worth answering first in this direction over the next few years?**

| Priority | Open question | Why it matters | Current evidence status |
|---|---|---|---|
| High | Whether open archive mechanisms can run in non-coding domains | Decides whether self-improvement equals "self-improving coding" | Nearly blank |
| High | Long-horizon stability (month / year scale) | Decides whether recursive self-modification is a sprint or a marathon | Only an eight-day short-cycle preprint |
| High | Unifying counter-evidence into a computable ceiling | Can predict how far a loop runs under given verifier quality | Three scattered works, not unified |
| Medium | Fair comparison of the three population architectures | Ends each side talking past itself | Each uses self-defined benchmarks |
| Medium | Mechanistic decomposition of context collapse | Affects all externalized-memory engineering practice | Named, not decomposed |

**Conclusion and boundary:** these five questions are, in turn, the boundaries of the existing evidence. It should be noted that "high/medium" priority is a subjective ordering by engineering impact and evidence gap; different positions may give different rankings.

At the end of the day, the key question for this direction is no longer "can the self-improvement loop run on coding benchmarks"—it already does. The real question is: over what time scale, how wide a task domain, and how deep a recursion level can this loop remain stable; and when it runs off course, whether we have good enough verifiers and guardrails to pull it back.

## References

1. Schmidhuber, J. (2003). Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. arXiv:cs/0309048. [https://arxiv.org/abs/cs/0309048](https://arxiv.org/abs/cs/0309048)

2. Wang, R., Lehman, J., Clune, J., & Stanley, K. O. (2019). Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions. arXiv:1901.01753. [https://arxiv.org/abs/1901.01753](https://arxiv.org/abs/1901.01753)

3. Zelikman, E., Wu, Y., Mu, J., & Goodman, N. D. (2022). STaR: Bootstrapping Reasoning With Reasoning. NeurIPS 2022; arXiv:2203.14465. [https://arxiv.org/abs/2203.14465](https://arxiv.org/abs/2203.14465)

4. Yuan, W., Pang, R. Y., Cho, K., et al. (2024). Self-Rewarding Language Models. ICML 2024; arXiv:2401.10020. [https://arxiv.org/abs/2401.10020](https://arxiv.org/abs/2401.10020)

5. Zhao, A., Wu, Y., Yue, Y., et al. (2025). Absolute Zero: Reinforced Self-play Reasoning with Zero Data. NeurIPS 2025 Spotlight; arXiv:2505.03335. [https://arxiv.org/abs/2505.03335](https://arxiv.org/abs/2505.03335)

6. Zweiger, A., Pari, J., Guo, H., et al. (2025). Self-Adapting Language Models (SEAL). NeurIPS 2025; arXiv:2506.10943. [https://arxiv.org/abs/2506.10943](https://arxiv.org/abs/2506.10943)

7. Shinn, N., Cassano, F., Berman, E., et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. ICLR 2023; arXiv:2303.11366. [https://arxiv.org/abs/2303.11366](https://arxiv.org/abs/2303.11366)

8. Wang, G., Xie, Y., Jiang, Y., et al. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291. [https://arxiv.org/abs/2305.16291](https://arxiv.org/abs/2305.16291)

9. Zhang, Q., Hu, C., Upasani, S., et al. (2025). Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (ACE). ICLR 2026; arXiv:2510.04618. [https://arxiv.org/abs/2510.04618](https://arxiv.org/abs/2510.04618)

10. Fernando, C., Banarse, D., Michalewski, H., Osindero, S., & Rocktäschel, T. (2023). Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution. arXiv:2309.16797. [https://arxiv.org/abs/2309.16797](https://arxiv.org/abs/2309.16797)

11. Hu, S., Lu, C., & Clune, J. (2024). Automated Design of Agentic Systems (Meta Agent Search). arXiv:2408.08435. [https://arxiv.org/abs/2408.08435](https://arxiv.org/abs/2408.08435)

12. Zelikman, E., Lorch, E., Mackey, L., & Kalai, A. T. (2023). Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. COLM 2024; arXiv:2310.02304. [https://arxiv.org/abs/2310.02304](https://arxiv.org/abs/2310.02304)

13. Yin, X., Wang, X., Pan, L., et al. (2024). Gödel Agent: A Self-Referential Agent Framework for RSI. ACL 2025; arXiv:2410.04444. [https://arxiv.org/abs/2410.04444](https://arxiv.org/abs/2410.04444)

14. Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025). Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. ICLR 2026; arXiv:2505.22954. [https://arxiv.org/abs/2505.22954](https://arxiv.org/abs/2505.22954)

15. Robeyns, M., Szummer, M., & Aitchison, L. (2025). A Self-Improving Coding Agent. arXiv:2504.15228. [https://arxiv.org/abs/2504.15228](https://arxiv.org/abs/2504.15228)

16. Wang, W., Piękos, P., Nanbo, L., et al. (2025). Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine. ICLR 2026; arXiv:2510.21614. [https://arxiv.org/abs/2510.21614](https://arxiv.org/abs/2510.21614)

17. Weng, Z., Antoniades, A., Nathani, D., et al. (2026). Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing. arXiv:2602.04837 (preprint, not peer-reviewed). [https://arxiv.org/abs/2602.04837](https://arxiv.org/abs/2602.04837)

18. Zhang, J., Zhao, B., Yang, W., et al. (2026). Hyperagents. arXiv:2603.19461 (preprint, not peer-reviewed). [https://arxiv.org/abs/2603.19461](https://arxiv.org/abs/2603.19461)

19. Romera-Paredes, B., Barekatain, M., Novikov, A., et al. (2023/24). Mathematical discoveries from program search with large language models (FunSearch). Nature 625:468–475 (online 2023-12-14, print issue January 2024). [https://www.nature.com/articles/s41586-023-06924-6](https://www.nature.com/articles/s41586-023-06924-6)

20. Lu, C., Lu, C., Lange, R. T., Foerster, J., Clune, J., & Ha, D. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292 (2024 preprint; the team's formal Nature version is a separate paper). [https://arxiv.org/abs/2408.06292](https://arxiv.org/abs/2408.06292)

21. Novikov, A., Vũ, N., Eisenberger, M., et al. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv:2506.13131. [https://arxiv.org/abs/2506.13131](https://arxiv.org/abs/2506.13131)

22. Ma, Y. J., Liang, W., Wang, G., et al. (2023). Eureka: Human-Level Reward Design via Coding Large Language Models. ICLR 2024; arXiv:2310.12931. [https://arxiv.org/abs/2310.12931](https://arxiv.org/abs/2310.12931)

23. Real, E., Liang, C., So, D. R., & Le, Q. V. (2020). AutoML-Zero: Evolving Machine Learning Algorithms From Scratch. ICML 2020; arXiv:2003.03384. [https://arxiv.org/abs/2003.03384](https://arxiv.org/abs/2003.03384)

24. Rank, B., Bhatnagar, H., Prabhu, A., et al. (2026). PostTrainBench: Can LLM Agents Automate LLM Post-Training? arXiv:2603.08640 (preprint, not peer-reviewed). [https://arxiv.org/abs/2603.08640](https://arxiv.org/abs/2603.08640)

25. Srikanth, D., Zhao, B. C., Xu, D., Wu, Y., & Jiang, Z. (2026). Recursive self-improvement of AI research agents (AIDE²). arXiv:2609.26457 (2026 preprint, short-cycle, no external reproduction). [https://arxiv.org/abs/2609.26457](https://arxiv.org/abs/2609.26457)

26. Huang, J., Chen, X., Mishra, S., et al. (2023/24). Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024; arXiv:2310.01798. [https://arxiv.org/abs/2310.01798](https://arxiv.org/abs/2310.01798)

27. Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2023/24). The Curse of Recursion: Training on Generated Data Makes Models Forget. Nature 631:755–759 (2024); arXiv:2305.17493 (2023). [https://arxiv.org/abs/2305.17493](https://arxiv.org/abs/2305.17493)

28. Wijk, H., Lin, T., Becker, J., et al. (METR) (2024). RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. arXiv:2411.15114. [https://arxiv.org/abs/2411.15114](https://arxiv.org/abs/2411.15114)

29. Shao, S., Ren, Q., Qian, C., et al. (2025). Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents. ICLR 2026; arXiv:2509.26354. [https://arxiv.org/abs/2509.26354](https://arxiv.org/abs/2509.26354)

30. Song, Y., Zhang, H., Eisenach, C., Kakade, S., Foster, D., & Ghai, U. (2024/25). Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models. ICLR 2025 Oral; arXiv:2412.02674. [https://arxiv.org/abs/2412.02674](https://arxiv.org/abs/2412.02674)

31. Yi Duan, et al. (2026). The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873 (2026 preprint, secondary survey, not first-hand evidence). [https://arxiv.org/abs/2609.11873](https://arxiv.org/abs/2609.11873)