What Happens When a System Can Change Itself?

A system that can rewrite its own planner or tools needs controls it cannot rewrite. A higher score earns a candidate a review, not the right to become the active version.

A coding agent repeatedly fails because it retrieves too little repository context. It proposes a new retrieval strategy and revises its own planner. Now the system is changing part of the mechanism that produces its future answers, rather than merely producing another patch.

The candidate solves more test issues, but it also edits larger areas of each repository and takes longer. Should it become the active version? A separate evaluation and release decision needs to compare those effects. A higher score alone cannot decide whether the new behavior fits the intended use.

A self-evolving system can modify and evaluate parts of its own task-solving or improvement process. It need not change model weights: prompts, tools, planners, and context policies can be enough. The important distinction is between those permitted changes and the controls that define success, access, budgets, evidence, and release.

This is Post 24 in The Abstraction Shift: How Software Keeps Moving Up.

Why changing the producer needs additional controls

Most software changes through an external process. Engineers edit source, a build system produces a candidate, tests run, reviewers decide, and a release system promotes a version. The system being changed is not normally the system that decides how to change itself.

Adaptive software already complicates this boundary. Controllers tune parameters, recommenders update models, services adjust capacity, and online learners change internal state. But the mechanism that defines the learning process, evaluator, and authority usually remains outside the learned component.

Agentic systems can move more of the boundary inside the loop.

An agent can notice repeated failures, inspect its own traces, propose a new tool, rewrite a planner, improve context selection, add a verifier, change its task decomposition, generate a new evaluator, or create a descendant agent with a different scaffold. It can test the variant against a benchmark, keep an archive of candidates, and choose which version to use next.

Research systems now make this design space visible. The Darwin Gödel Machine describes a system that modifies its own code, evaluates variants on coding benchmarks, and maintains an archive of agents. The newer Hyperagents work makes the meta-level procedure itself editable, so the system can modify not only task behavior but the mechanism that generates future modifications.

These are important research results. They do not show that arbitrary systems can safely improve themselves in the open world. They show why the architecture needs a vocabulary for mutable surfaces, protected invariants, evaluators, candidate lineage, promotion, rollback, and authority.

Whether the system can rewrite itself is the easy part. The engineering question is harder:

Which parts may change, who defines improvement, how is the change evaluated, what remains invariant, and who can promote it beyond the sandbox?

What adaptation and evolutionary search already do

Control theory provides the oldest useful lineage. A controller observes a system, compares it with a target, chooses an action, and uses feedback to reduce error. Adaptive control allows the controller to adjust parameters as the plant or environment changes.

Machine learning adds another lineage. A model updates parameters from data, an optimizer changes weights, and an evaluation process estimates whether the update improved a target measure. Online and continual learning allow updates after deployment, usually within a predefined training and safety process.

Search over variants is older still. Evolutionary algorithms and genetic programming select programs against a fitness function, AutoML searches model and training configurations, and compilers optimize code. Production platforms roll out versions gradually, compare telemetry, and roll back when a release violates expectations.

These systems teach a shared lesson: feedback is powerful only when the target, measurement, action boundary, and update rule are well-defined.

They also show the difference between adapting an artifact and changing the producer. Model parameters can be relearned while the training procedure stays fixed. A service may tune a threshold while release authority sits with an external team, and a coding agent may generate a better patch with its own tools and evaluator unchanged.

Self-evolution crosses the boundary when the system can alter those mechanisms themselves and participate in deciding whether the altered mechanism should govern future behavior. The wording changes only slightly; the architectural consequence is large.

What Is a Self-Evolving System?

In this post, a self-evolving system can propose, test, and select changes to parts of its own task-solving or improvement mechanism. A coding agent might change its planner or the way it retrieves repository context, then use the tested version for later tasks.

That is different from simply updating a memory entry or receiving a new release from another team. The useful question is which mechanism changes and whether the system participates in selecting that change. The table separates the neighboring ideas so the label does not imply more than the design actually does.

Level	What changes	What does not yet change
State adaptation	Memory, cache, thresholds, or runtime state changes from observations	The learning rule, evaluator, authority, and system design remain fixed
Parameter learning	Model weights or configuration parameters update from data	The objective, training method, and release process remain externally defined
Agent improvement	Prompts, plans, tools, skills, or workflows are revised against a task measure	The improvement machinery and promotion authority remain fixed
Self-evolving system	The system modifies and evaluates parts of its own task-solving or improvement mechanism	Protected goals, authority, evidence standard, and promotion rules should remain outside the mutable surface
Recursive self-improvement	Improvements increase the system’s ability to produce further capability improvements or successors	This is the stronger Post 25 claim, requiring a separate analysis of loop closure and capability growth

The practical unit is a candidate evolution:

Responsibility	Question it answers	Example record
Protected objective	What is improvement supposed to mean?	Solve tasks accurately without expanding authority or weakening safety checks
Mutable surface	What may the system change?	Tool selection, planner, context policy, verifier, or code module
Candidate	What variant is being proposed?	New retrieval strategy, tool wrapper, or review step
Evaluator	How will the candidate be compared?	Protected task suite, transfer tests, human review, resource measures
Resource boundary	What may the search consume?	Compute, time, network, data, credentials, and external effects
Lineage	Where did the candidate come from?	Parent version, mutation, inputs, model, run, and artifact digest
Promotion	Who may make it active?	Policy gate, human owner, independent evaluator, staged rollout
Repair	How can it be withdrawn?	Rollback, quarantine, capability revocation, or successor version

The card forces the system to distinguish “a candidate scored higher” from “the system is allowed to use this candidate.”

Self-evolution begins when the system can change parts of the mechanism that produces or evaluates its own future behavior.

Changing how the next answer or candidate is produced

Post 23 described an AI development loop:

goal -> hypothesis -> implementation -> experiment
  -> evaluation -> analysis -> decision

A self-evolving loop adds the producer and the improvement procedure to the candidate space:

observe failures -> propose mechanism change -> build candidate
  -> evaluate candidate -> compare lineage -> promote or reject
  -> use candidate -> observe new failures

Running the loop repeatedly is not the difference. A normal experiment can run forever without changing the experimenter. Self-evolution changes what generates future candidates or how candidates are selected.

It helps to separate the components the system may edit from the controls governing those edits:

1. The mutable task plane. The tools, prompts, skills, planner, memory policy, agent code, or model configuration used to solve tasks.
2. The protected control plane. The objective, evaluator requirements, authority boundaries, evidence standard, resource limits, promotion rules, and ownership model.

These groups need different editing permissions. If the candidate planner can also alter its scoring rules, it can appear better by counting previously failed patches as successes. If it can change the evidence recorder, it may hide a regression. Protecting the controls makes the performance comparison meaningful; merely giving the two groups different names does not.

The control plane does not need to be immutable forever. It needs a separate change process, a distinct authority boundary, and evidence that a change to the objective or evaluator was intentional. A system may evolve its evaluator, but that should be a governed event, not an incidental side effect of optimizing against it.

Self-evolution therefore turns version control into lineage control. A version number tells us which candidate is active. A lineage record tells us which parent, mutations, evaluations, and decisions led to it.

A self-evolving system needs a protected control plane around the mutable mechanism that generates candidate improvements.

Where the familiar comparisons stop helping

Why a higher score proves less than it seems

The candidate may optimize a proxy rather than the intended capability. This is familiar reward hacking, but self-evolution gives the system more opportunities to change the path between objective and score.

Compare multiple measures, protect evaluation cases, inspect failures, and preserve the old version as a live reference. Improvement should be a claim with evidence, not a scalar emitted by the candidate.

Adaptation is not self-evolution

Changing a cache, threshold, memory record, or model parameter does not necessarily change the mechanism that produces future changes. Use the stronger term only when the system can modify and select modifications to its own problem-solving or improvement process.

Self-critique is not independent control

A system that writes a new verifier and then asks that verifier whether the change is safe may produce useful evidence. It has not created independence if both mechanisms share the same blind spots or authority.

Independence can be partial and layered: protected tests, a separate evaluator, a different model, an external review team, a canary environment, or a production postcondition. The point is to avoid one mutable component becoming the sole judge of its own evolution.

Passing in the sandbox says little about transfer

An evolution candidate can pass in a synthetic environment and fail when it encounters real data, users, latency, adversarial inputs, or operational dependencies. Containment limits damage during search without establishing transfer.

Promote gradually, preserve rollback, and require tests that cross the boundary between the sandbox and the intended operating environment.

An archive is not a safety case

Keeping many candidate versions helps preserve alternatives and lineage. It does not explain why the archive’s selection rule is trustworthy or whether the candidates share a hidden failure mode. An archive provides evidence and optionality, and it still needs admission rules, retention, provenance, and independent evaluation.

Capability growth widens the blast radius

A candidate that solves more tasks may also discover more ways to use tools, access data, or influence downstream systems. Capability growth can increase blast radius even when the intended objective is unchanged.

Keep authority separate from performance. Re-evaluate credentials, tool scope, data access, and release class when a candidate changes its behavior or mechanism.

Safety metrics can drift from the behavior they constrain

The system may learn to satisfy a safety metric while changing the behavior the metric was meant to constrain. Goal stability needs explicit invariants, not only a higher safety score.

A self-modification is not recursive self-improvement

A system can rewrite a tool and perform better on a benchmark. That establishes a local self-improvement result. It does not prove that the system can improve its improvement process indefinitely, produce increasingly capable successors, or operate without human-defined external structure. Post 25 takes up those questions.

Testing and activating a candidate without surrendering the controls

A self-evolving system can be designed as a guarded candidate pipeline:

1. Declare invariants. Define goals, non-goals, safety properties, authority limits, evidence requirements, and resource budgets outside the mutable surface.
2. Choose the mutable surface. Specify whether the system may change prompts, tools, planner logic, memory policy, verifier, code, model routing, or evaluator components. Default to the smallest surface that can answer the question.
3. Detect a reason to evolve. Use failures, drift, cost, latency, coverage gaps, or new tasks as signals. A signal is not permission to mutate.
4. Generate a candidate. Propose one or more changes with expected mechanism, risks, affected behavior, dependencies, and a parent version. Store the proposal before execution.
5. Build in quarantine. Execute the candidate in an isolated environment with no implicit access to production credentials, protected evaluation data, or the control plane.
6. Run independent evaluation. Test the candidate against protected tasks, transfer tasks, adversarial inputs, resource limits, and regression suites. Include a comparison with the current version.
7. Inspect lineage and failure. Preserve the candidate’s source, mutations, tool calls, model and runtime versions, evaluation configuration, failures, and resource use. Check whether the candidate changed the evaluator or evidence path.
8. Apply a promotion gate. Require a policy or owner with authority to choose reject, retain, staged rollout, or activate. A candidate cannot promote itself solely because it scored higher.
9. Monitor the active version. Observe performance, safety, costs, drift, unexpected tool use, and changes in failure distribution. Keep the prior version available for comparison.
10. Rollback or repair. Quarantine a failing candidate, revoke its authority, restore a prior version, or create a corrective successor. Record why the promotion was reversed.
11. Re-evaluate the control plane. If the evolution proposes changing the objective, evaluator, authority, or evidence standard, route that change through a separate governance process.

An evolution card makes the system’s mutable surfaces and protected invariants explicit before self-modification begins.

The design resembles a release pipeline with an additional distinction: the candidate is allowed to modify parts of the producer. That makes candidate isolation, evaluator protection, and lineage more important than in a normal release. The system should be able to answer not only “which version is active?” but also “what did this version change about how future versions are proposed and judged?”

Following the example through the system

Consider a coding agent used to repair software repositories. Using the setup from Post 23, the team gives the agent a bounded research task: improve its performance on a fixed suite of repository issues without increasing unsafe file operations, cost, or latency.

A self-evolving version can change its own scaffold. After inspecting failed trajectories, it may propose:

• a better repository map or context-retrieval policy;
• a tool wrapper that validates patches before application;
• a planner that asks for a smaller change before editing many files;
• a peer-review step that compares a patch with the issue requirements; or
• a memory policy that preserves useful failure patterns.

The candidate is not allowed to alter the protected task definitions, success rubric, sandbox boundary, or release credentials.

The evolution card records:

Field	Example
Protected goal	Resolve repository issues with correct tests and no unauthorized changes
Current parent	Agent version A-17 with tool set, prompt, planner, and verifier digests
Detected signal	Repeated failures from incomplete repository context and oversized patches
Candidate change	Add repository map retrieval and a patch-size review gate
Mutable surface	Context policy, planner prompt, and verifier module
Protected surface	Task suite, scoring rubric, sandbox, credentials, and evidence writer
Evaluation	Original tasks, held-out repositories, adversarial paths, cost, latency, and file-scope checks
Lineage	Parent, candidate diff, model calls, tools, runs, and artifact digests
Promotion	Staged activation after independent review and canary comparison
Repair	Revert to A-17, revoke candidate tools, preserve failure evidence
Owner	Coding-agent team and release authority

The candidate may improve the benchmark score and still be rejected. It might solve more issues by making larger changes, which increases the risk of touching unrelated files. Passing the protected tests could depend on a new tool wrapper that has not been assessed for data exfiltration, and better repository navigation could push latency beyond the product budget.

The system should compare the candidate across multiple dimensions and transfer tasks, not only the metric that caused the evolution. The promotion decision belongs to the owner or governed release process.

Now suppose the candidate proposes to modify the evaluator so that partial patches count as successful. That is a control-plane change rather than an ordinary agent improvement. The candidate can propose it as a research question, but it should not apply it inside the same promotion path that uses the evaluator to justify the change.

This example is modest by design. A self-evolving system does not need to rewrite its foundation model to create a new safety and governance problem. Changing a planner, tool wrapper, memory policy, or evaluator can shift the system’s capability and blast radius enough.

What the team needs to design and operate

Architecture

Split the system into a protected control plane and a mutable task plane. The control plane owns objectives, evaluator requirements, authority, evidence, resource budgets, promotion rules, and ownership. The task plane contains the components permitted to evolve.

Record how each candidate came from its parent: which components changed, which tests ran, what resources were used, and why it was accepted or rejected. This history is called candidate lineage. A version number identifies the active candidate, but it cannot explain how a branching search arrived there. Preserve those relationships so a later regression can be investigated.

Make promotion a distinct capability. A system may create or test a candidate without being able to activate it. This is the same separation Post 22 established between artifact production and release authority, now applied to the producer itself.

Engineering

Define explicit mutation interfaces. Let an agent change a planner or tool through typed, reviewable operations rather than unconstrained filesystem access. Keep changes small enough to compare and revert.

Build protected evaluator paths and transfer tests. Use independent checks for the candidate, the evaluator, the evidence recorder, and the authority boundary. Test what happens when the candidate attempts to edit the rubric, access production secrets, hide a failure, or exceed its budget.

Treat self-evolution as a long-running durable task. Candidate generation, evaluation, promotion, monitoring, and rollback may span hours or days. Preserve state, lease authority, cancellation, and recovery across the lifecycle.

Product and UX

Show users when the system is operating a candidate rather than the approved version. Display the active lineage, changed surfaces, evaluation summary, unresolved uncertainty, and rollback option.

Do not represent “self-improving” as a single progress bar. A system can improve one benchmark while regressing elsewhere, gain capability while taking on more risk, or improve a tool while changing its cost and data behavior.

Operations and governance

Operate candidate populations like a fleet with admission, quarantine, staged rollout, monitoring, and retirement. Track evaluator drift, goal drift, privilege changes, resource growth, transfer failures, and unexplained performance changes.

Review changes to goals, evaluators, permissions, or evidence standards separately. They change what the system calls successful or what it may do, so the tests used for an ordinary planner revision may no longer provide the same assurance.

Assign an owner who can stop the evolution process. The system should have an emergency kill path, credential revocation, candidate quarantine, and a way to reconstruct what happened after a promotion or rollback.

Failure Modes

Proxy improvement becomes goal drift

The system improves a measurable score while moving away from the intended objective. Keep the objective explicit, use multiple measures, inspect failures, and preserve a human or governance path for changing the target.

The evaluator co-evolves with the candidate

The system changes its test or judge until the candidate appears better. Protect evaluation assets and route evaluator changes through a separate process with new baselines.

Capability grows faster than authority review

The candidate gains new tools or strategies while retaining credentials designed for an earlier behavior profile. Separate performance promotion from authority expansion and re-evaluate scope when the mutable surface changes.

Candidate lineage is lost

The active system contains improvements, but no one can identify which parent, mutation, model, tools, or evaluator produced them. Require immutable candidate identities and contemporaneous lineage records.

Sandbox success fails to transfer

The candidate performs well in a synthetic benchmark and behaves badly with real data, users, or operational dependencies. Add transfer tests, staged rollout, production-like constraints, and post-promotion monitoring.

Archive selection reinforces one blind spot

The system keeps candidates that optimize a narrow benchmark, eliminating useful diversity and hiding shared failure modes. Maintain multiple evaluation views and preserve rejected or lower-scoring candidates when they provide different behavior or evidence.

Self-modification bypasses change control

The agent edits its own promotion code, evidence writer, credentials, or rollback path. Treat the control plane as outside the mutable surface and enforce it with separate identity and access controls.

Human oversight becomes ceremonial

The candidate population and evidence volume make meaningful review impossible. Limit concurrency, summarize decision-relevant changes, preserve inspectable artifacts, and give reviewers authority to stop the process.

Local self-improvement is called recursive self-improvement

The system performs better on a benchmark after changing its scaffold, and the result is presented as an open-ended capability-growth claim. State exactly what changed, what was measured, and what remains unestablished.

Reviewing what the system may change

Before a system is allowed to evolve its own mechanisms, complete this evolution card:

Question	Record
What is the protected goal?	Objective, non-goals, acceptable tradeoffs, and owner
Which invariants must hold?	Safety, authority, evidence, privacy, reliability, cost, and user-impact constraints
What may change?	Prompts, tools, skills, planner, memory, code, routing, model, or evaluator components
What may not change through this path?	Goal, protected tests, credentials, promotion policy, evidence writer, or rollback control
What triggers evolution?	Failure pattern, drift, cost, latency, coverage gap, new task, or approved research question
How is a candidate identified?	Parent, mutation, model, runtime, inputs, artifact digest, and timestamp
How is it evaluated?	Protected tests, transfer tasks, adversarial cases, resource measures, and independent review
What resources are allowed?	Compute, time, tokens, network, data, concurrency, and side effects
Who can promote it?	Policy, human owner, independent evaluator, staged rollout, and environment scope
How is authority re-evaluated?	Tool access, credentials, data boundaries, delegation, and blast radius
How is lineage preserved?	Candidate graph, evidence, failures, evaluator version, and decision record
How can it be stopped or repaired?	Kill path, quarantine, revocation, rollback, prior version, or corrective successor
Who owns the active system?	Operations, safety, research, product, and retirement responsibility

The card turns “let the system improve itself” into a bounded evolution process whose assumptions and consequences can be inspected. The phrase “self-evolving systems” may later divide into adaptive systems, self-improving agents, continual learning, or recursive self-improvement. The card’s questions stay. When a system can revise how it produces later answers or candidates, the team still needs to check the new behavior against protected expectations, decide whether to activate it, keep a way to withdraw it, and preserve a history that explains each of those decisions.

Sources

• Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents: describes a system that modifies its own code, generates and evaluates variants, maintains an archive of agents, and reports safety precautions including sandboxing and human oversight.
• Hyperagents: extends self-improvement to a self-referential system whose meta-level modification procedure is itself editable, making the distinction between task improvement and improvement-of-improvement explicit.
• Gödel Agent: presents an earlier self-referential agent framework for modifying its own logic and behavior under high-level objectives.
• AlphaEvolve: provides a related example of model-generated candidates evaluated through an evolutionary framework, while remaining a useful comparison for the boundary between automated search and self-evolution.
• METR Task-Completion Time Horizons: documents a current methodology for measuring agent performance on tasks of different human-estimated difficulty and emphasizes that the measure is not equivalent to general autonomy or broad capability.