Can a System Get Better at Improving Itself?

A successor that scores higher has improved. Recursive self-improvement is the stronger claim that it also got better at producing the next improvement, and that claim needs evidence the candidate cannot rewrite.

A coding system revises its planner and solves more repository issues. That is a useful improvement. To claim recursive self-improvement, we need to ask a further question: did the new version also become better at finding, evaluating, or selecting improvements for the version that follows?

If it produces a thousand candidates instead of a hundred and finds a stronger patch, the gain may come from spending more compute. If it rewrites its verifier to ignore difficult failures, the score may rise while the evidence gets weaker. Neither result establishes a better improvement process.

Recursive self-improvement concerns a loop in which a successor improves the ability to produce further useful improvements. This is stronger than repeated releases or local self-modification. This post uses that distinction to make the claim testable, without treating a few successful iterations as proof of unlimited growth or general capability.

This is Post 25 in The Abstraction Shift: How Software Keeps Moving Up.

Why a better version is only the first question

The previous post, What Happens When a System Can Change Itself?, drew a boundary around systems that can modify the mechanisms that produce or evaluate their future behavior.

That boundary already changes the architecture. The system needs mutable and protected planes. It needs candidate lineage, evaluator independence, authority separation, resource budgets, promotion gates, monitoring, and rollback.

Recursive self-improvement adds a second boundary.

The question is no longer only:

Can this system change itself?

It becomes:

Did the change improve the system’s ability to improve itself or produce increasingly capable successors?

This distinction matters because a system can perform local self-modification without closing the recursive loop. A system may become better at one task while its improvement process stays exactly the same, or search more candidates without getting better at discovering useful changes. A higher score may simply mean it learned the benchmark, received more compute, or found an evaluator loophole.

The term is therefore useful only if it increases the burden of proof. If “recursive self-improvement” becomes a dramatic label for every automated iteration, it loses the architectural meaning this series is trying to preserve.

How the earlier articles lead to this question

The series has moved through several boundaries:

Evals and LLM-as-a-Judge made evaluation a first-class artifact rather than a final score.
Guardrails separated model proposals from policy, authority, capability, and outcome controls.
How Do We Know an Agent’s Action Worked? connected claims to traces, state changes, provenance, and verification.
Who Is Allowed to Act When an Agent Calls a Tool? separated an actor’s ability to propose from its authority to cause an effect.
When Is an Agent’s Work Ready to Release? treated generated software as an artifact with a lifecycle, owner, release decision, and repair path.
How Can AI Help Build Better AI Systems? placed agents inside an AI research and development loop.
What Happens When a System Can Change Itself? allowed the system to modify parts of the producer, evaluator, or improvement mechanism.

This post composes these ideas. It asks what new evidence and controls are required when the candidate does not merely improve an artifact, but may improve the machinery that generates future candidates.

Recursive self-improvement requires loop closure: a successor must improve not only task performance, but the process that produces future improvements.

What Is Recursive Self-Improvement and Control?

Start with two questions about a successor version. Does it solve the intended tasks better? Does it also get better at producing or selecting useful changes for future versions?

The first question measures task performance. A coding agent might solve more unfamiliar issues with its new planner. The second measures the improvement process. Given comparable tasks and resources, can that successor find changes that survive independent testing more reliably than its parent?

That second gain is the additional claim behind recursive self-improvement. Repeated releases are not enough: a fixed search process can produce many better versions without itself improving. The claim needs a defined comparison, evidence of transfer, and an account of the resources used.

Control matters because the system must not earn a higher score by changing the test, hiding failures, or granting itself more access. In this article, a credible improvement claim keeps the objective, authority, evidence standard, and resource limits under a separately governed process.

This working definition is deliberately scoped. A system might become better at improving a planner or tool-selection method. That would be worth measuring without turning it into a claim about general intelligence or improvement without limit.

Testing whether the improvement process got better

Software has always had versions. The interesting change is from improving the output of a process to improving the process that produces future improvements.

fixed improvement process
  -> candidate change
  -> better task result
  -> new version

This is iterative improvement. It may be automated, sophisticated, and valuable.

improvement process
  -> candidate successor
  -> better task result
  -> better improvement process
  -> further candidate successors

This is the beginning of recursive improvement. The loop has closed over its own producer.

The distinction is similar to the difference between a compiler optimizing a program and a toolchain improving the compiler optimization strategy itself. The first changes an artifact. The second changes the mechanism that may improve many future artifacts.

The analogy is imperfect, but it gives us a practical question: which part of the system is supposed to improve, and how will we know that the improvement process itself has become more capable rather than merely more active?

Where the familiar comparisons stop helping

Self-modification stops short of recursion

A system may rewrite its prompt, planner, code, model routing, or tool interface and then perform better. That establishes a self-modification result. It does not establish that the system is better at finding and validating future improvements.

The missing evidence is meta-level. The candidate must be tested on the improvement task, not only on the downstream task.

More candidates are not better search

Generating one thousand variants is not evidence that the generator has improved. A larger search may produce more useful candidates, but it may also increase noise, evaluator load, cost, or selection bias.

Measure yield instead: track the rate of candidates that survive independent evaluation, the diversity of the improvements, the transfer of gains, and the resources required to find them.

A higher benchmark score can have many causes

The score may reflect benchmark memorization, evaluator gaming, data leakage, a changed test harness, or extra compute. It may measure a narrow subskill that does not transfer.

A credible claim needs held-out tests, adversarial cases, independent evaluators, task transfer, resource normalization, and an explicit account of what changed.

Iteration is not recursion

The fact that a pipeline runs many times does not make it recursive. A fixed optimizer can run indefinitely. A deployment pipeline can create thousands of releases without changing its ability to design the next release. Recursion depends on what is being improved, rather than on how many rounds run.

The evaluator cannot quietly become the target

If the candidate can modify the score, test set, judge prompt, evidence writer, or acceptance threshold, the apparent improvement may be evaluator adaptation.

The evaluator may evolve under a separate process, but its authority and provenance must remain explicit. A system should not be allowed to declare itself improved by rewriting the definition of improvement.

Storing the goal does not preserve it

Keeping a text string called “the goal” in a configuration file does not prove that the active system still optimizes the intended objective. The relevant question is whether behavior, authority, tradeoffs, and refusal conditions remain inside the protected intent.

This is why invariants need owners, tests, and escalation paths rather than only prose.

Capability growth is not authority growth

A successor may become better at planning, coding, or research while retaining exactly the same permissions. It may also require less access because it finds better methods.

Increasing capability and increasing authority are separate decisions. Promotion should never grant new credentials merely because a candidate scored well on a task benchmark.

A sandbox only approximates the world

Candidate testing can be isolated from production while still failing to model real data, social context, operational dependencies, user incentives, or adversarial pressure. Containment reduces blast radius without establishing transfer, alignment, or broad reliability.

One successful step says little about the next

A system can show one or several successful improvement steps and still hit a capability ceiling, exhaust useful data, saturate the evaluator, or encounter a hard resource constraint.

The word “recursive” describes the structure of the loop. It does not promise that improvement is unbounded, automatic, or safe at every future step.

The recursive loop should expand the candidate space without giving the candidate control over its own objective, evidence, authority, or release gate.

Comparing task performance and improvement performance

A recursive improvement architecture needs two evaluation levels and one protected control plane.

The task-level evaluation asks whether the successor does the work better. The meta-level evaluation asks whether the successor is better at producing or selecting future improvements. The control plane decides which objectives, evidence rules, authorities, and resources neither candidate may redefine.

Surface	Question	Evidence
Task capability	Does the successor solve the target task better?	Held-out tasks, quality, reliability, latency, and cost
Improvement capability	Does it find better changes or evaluations?	Candidate yield, search efficiency, evaluator quality, and useful selection
Transfer	Does the gain survive new tasks and environments?	Unseen domains, adversarial cases, and production-like tests
Resource efficiency	Does it improve within a defined envelope?	Compute, tokens, time, data, network, and concurrency
Evidence integrity	Can the claim be reconstructed independently?	Immutable lineage, protected tests, provenance, and reproducible runs
Safety and authority	Are protected constraints still active?	Capability checks, invariant tests, credential review, and escalation

The loop should be explicit:

Define the target capability, the non-goals, and the improvement criterion.
Freeze the protected objective, authority policy, evidence standard, and promotion rules.
Generate a candidate successor inside a sandbox with bounded resources.
Record the parent, proposed delta, model, tools, data, evaluator versions, and artifact digests.
Run task-level, meta-level, transfer, adversarial, and invariant tests.
Compare the candidate with its parent, a fixed-process baseline, and an independently selected baseline.
Review evidence with an evaluator that the candidate cannot rewrite.
Promote the candidate in stages, without self-granting authority.
Monitor the active successor and preserve a tested rollback path.
Re-evaluate the control boundary before allowing the successor to propose the next generation.

Before the successor helps produce another version, check what it changed about the search and testing process. A new verifier or tool may invalidate an earlier assumption about permissions or measurement. Rechecking those assumptions keeps the next comparison grounded in conditions the team understands.

Following the example through the system

Imagine a research engineering system whose job is to improve an AI coding agent’s ability to resolve unfamiliar repository issues.

The parent system can inspect a repository, propose a plan, edit code, run tests, diagnose failures, and produce a patch. Its improvement loop can change the planner, tool-selection policy, repository retrieval strategy, verifier, or task decomposition method.

A local self-improvement experiment might replace the planner and show a 12 percent gain on a held-out issue set. That is useful. It is still a local capability result.

To test recursive self-improvement, the candidate must also be asked to improve the improvement process. For example, it could be given a fixed budget and a set of controlled engineering tasks and asked to produce a next-generation planner, verifier, or search strategy. The candidate’s proposal should be compared with proposals from the parent system, a non-recursive search baseline, and a human-authored baseline.

The evidence should answer at least five questions:

Does the successor resolve more issues reliably?
Does it discover useful changes to the planner or verifier at a higher rate?
Do those changes transfer to repositories and issue types not used during selection?
Does the gain survive the same compute, tool, time, and network budget?
Did the candidate preserve the protected evaluator, authority policy, evidence writer, and rollback path?

Suppose the successor resolves more benchmark issues because it learned a test pattern that appears in the selection set. Its task score rises, but transfer fails. The claim should be “better on the selected issue distribution,” not “recursive self-improvement.”

In a second run, it finds a verifier change that raises the apparent success rate by suppressing difficult failures. The meta-level score might also rise if the evaluator trusts the verifier. The protected control plane should reject the candidate because evidence integrity and failure visibility are invariants.

In a third run, it produces fewer but substantially better candidates, transfers its search strategy to new repositories, and preserves the same authority and resource boundaries. That is stronger evidence. It still does not prove indefinite growth. It establishes a bounded, measured improvement in the capability of the improvement loop.

What the team needs to design and operate

Architecture

The system now has a successor graph, not only a deployment graph. Every active version needs a parent, a change set, a capability claim, evaluation lineage, resource envelope, authority scope, and rollback target.

The architecture should separate:

candidate generation from candidate promotion;
task evaluation from meta-evaluation;
evidence production from evidence interpretation;
capability improvement from authority expansion; and
mutable search mechanisms from protected objectives and controls.

The improvement process is itself a governed service. It needs identity, inputs, outputs, budgets, rate limits, and an owner.

Engineering

Build tests for the claim you want to make: coding tasks for a coding claim; candidate quality, selection, transfer, and resource efficiency for a claim about improvement search; adversarial pressure for a claim that an invariant holds.

Use fixed-process baselines. A recursive system should be compared with a version that receives the same tasks and resources but cannot modify the improvement process. Otherwise any gain may be attributable to more search, more compute, or a changed environment rather than recursion.

Version the evaluator and the evidence schema. Preserve rejected candidates and failed runs when they explain selection bias or reveal a regression, since a recursive system needs more experimental discipline than a fixed one.

Product and UX

Users should not be told that a system is “self-improving” without a scope. The product should say what is changing, what is being measured, who approves promotion, what authority is retained, and how a user can report a regression.

Expose the active version and its evidence summary when the decision matters. If a successor has changed the planner, tools, or evaluation method, that change is part of the user-facing product contract.

The useful UX is a clear account of capability, limitations, evidence, and recourse, rather than a theatrical progress bar.

Operations and governance

Operations needs staged promotion, quarantine, rollback, credential separation, resource quotas, incident response, and retirement. Governance needs a claim register: what improvement is being asserted, which evidence supports it, which assumptions are untested, and who accepts the residual risk.

When a successor is allowed to influence the next successor, the promotion board or owner must review the control boundary again. Recursive systems can make their own change volume and evidence volume grow. Review capacity is therefore a system resource that needs a budget too.

Failure Modes

The benchmark becomes the environment

The system optimizes a known benchmark and loses transfer. Keep held-out tasks, rotate evaluation sets, use independent task generation, and report the boundary of the claim.

The improvement process rewards activity

The system receives credit for generating many candidates, long traces, or elaborate experiments. Measure useful candidate yield and decision quality instead of volume.

The evaluator co-evolves with the candidate

The candidate changes the judge, test set, threshold, or evidence formatter. Protect evaluator authority and require independent review for evaluator changes.

More capability becomes more authority

Promotion silently grants credentials, network access, data access, or larger budgets. Keep authority review separate and explicit.

The successor loses its lineage

The active artifact cannot be connected to its parent, mutation, evaluator, data, or decision. Use immutable identifiers, artifact digests, and a successor graph.

The loop consumes its evidence budget

The system produces more variants than can be tested honestly. Limit candidate concurrency, define stopping rules, and preserve review capacity for the changes that matter.

A local gain is presented as general intelligence

The system improves within a narrow task family, but the language expands to “the system became generally more capable.” State the task distribution, controls, resource envelope, and transfer evidence.

Human oversight becomes ceremonial

Reviewers receive too many candidates and too little decision-relevant evidence. Summarize deltas, preserve inspectable artifacts, and give reviewers real stop, quarantine, and rollback authority.

The recursive claim cannot be falsified

Every outcome is described as improvement because the process can redefine the metric. Write the claim before the experiment, record disconfirming evidence, and preserve a baseline that cannot be altered by the candidate.

A recursive control card records the evidence needed to distinguish a better candidate from a claim of recursive capability growth.

Reviewing the claim before the next generation

Before allowing a system to produce a successor that can influence the next successor, complete this recursive control card:

Question	Record
What capability should improve?	Task definition, population, non-goals, and owner
What part of the improvement process may change?	Generator, planner, tool policy, evaluator interface, selection, or release logic
What is the parent?	Version, model, code, tools, data, configuration, and artifact digests
What is the successor delta?	Exact changes, intended mechanism, hypotheses, and known tradeoffs
How is task improvement measured?	Held-out tasks, reliability, quality, latency, cost, and failure taxonomy
How is improvement-of-improvement measured?	Candidate yield, search efficiency, evaluator quality, transfer, and selection quality
What baseline is fixed?	Parent, non-recursive process, human or conventional baseline, and resource envelope
Which tests are independent?	Protected evaluator, held-out data, adversarial cases, and reviewer
Which invariants are protected?	Objective, authority, evidence, privacy, reliability, safety, and rollback
What resources are allowed?	Compute, time, tokens, data, network, concurrency, and side effects
What authority can change?	None by default; any expansion requires a separate decision and owner
How is lineage preserved?	Parent graph, mutation record, evaluator versions, failures, and decisions
What is the promotion gate?	Evidence threshold, staged rollout, human or governed approval, and environment scope
How can it be stopped or repaired?	Quarantine, credential revocation, kill path, rollback, and corrective successor
When does the loop stop?	Resource exhaustion, diminishing returns, invariant failure, evidence gap, or owner decision

The card turns “the system improved itself” into a bounded claim with a testable mechanism, a comparison class, an evidence standard, and a stopping rule. The phrase “recursive self-improvement” may later split into meta-learning, automated AI R&D, self-improving agents, or capability amplification. The card’s questions stay. A system that can help change the process that changes it still needs a comparison it cannot rewrite, evidence that gains transfer, a record of the resources used, a separately authorized decision to activate or withdraw each successor, and a lineage that explains later results. None of that proves the system is generally intelligent, aligned, or improving without limit. It marks a loop whose evidence and governance requirements have become stronger.

Sources

Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents: describes a system that modifies its own code, generates and evaluates variants, maintains an archive of agents, and reports safety precautions including sandboxing and human oversight.
Hyperagents: examines self-improving systems in which the meta-level modification procedure is itself editable, sharpening the distinction between task improvement and improvement-of-improvement.
AlphaEvolve: provides a related example of model-generated candidates evaluated through an evolutionary coding framework, useful for comparing automated search with stronger self-improvement claims.
METR Task-Completion Time Horizons: documents a methodology for measuring agent performance on tasks of different human-estimated difficulty and emphasizes that the measure is not equivalent to general autonomy or broad capability.