Survey companion & reading list

A curated collection of research on self-evolving agents, advancing reliable AI self-improvement.

Paper (PDF) OpenReview Code / Reading List Explore L0–L4
549 manuscript-used works
5 self-evolution levels
8 catalog sections

Kaiqi Wang1,*,‡, Wenjin Hou1,2,*, Yuchen Yan1,2,*, Hongrui Jia1,3,*, Zhisheng Zhong1,*, Botao Ren1,4,* Yifei Chen1, Songyang Zhang1,§, Yongliang Shen2, Jun Xiao2, Yi Yang2, Yueting Zhuang2, Hehe Fan2,†

* Core Contributors ‡ Project Lead § Project Supervisor † Corresponding Author

1 Tencent Hunyuan 2 Zhejiang University 3 Peking University 4 Tsinghua University

§ Abstract

Diving into Reliable Self-Evolving Agents: A Survey

Self-evolving agents use information produced during their own execution to revise current outputs or modify retained agent components that shape later behavior and future updating. As these systems gain broader and more persistent self-modification capabilities, reliability becomes a central concern. Two questions guide this survey: what changes during self-evolution, and what evidence can support claims of improvement. The literature is organized by self-evolution depth, defined by the deepest evolution target whose change takes effect—from the current output to retained components that shape future behavior, updates, or judgments. Accordingly, we classify self-evolution into five levels: Output-Level Self-Evolution (L0), Model-Level Self-Evolution (L1), Scaffold-Level Self-Evolution (L2), Improver-Level Self-Evolution (L3), and Criterion-Level Self-Evolution (L4). For each level, we survey representative systems and compare their evolution mechanisms, changed objects, persistence conditions, and reliability challenges. We then provide a cross-level synthesis of these concerns and develop a reliability ladder that pairs each evolution target with evidence and controls outside the corresponding update boundary. Reliable self-evolution thus depends not on self-evolution depth alone, but on whether evaluation and oversight remain independent of the update and cover the relevant tasks, conditions, and constraints. Finally, we discuss key challenges and future research directions. We aim for this survey to serve both as a structured reference for existing work and as guidance for developing the next generation of capable and reliable self-evolving agents.

Keywords self-evolving agents; recursive self-improvement; reliability; self-evolution depth; external audit; reliability ladder

◇ Landscape

What changes during self-evolution?

Five levels show how deeply a change reaches into the agent—from a current output to the criteria that judge future updates.

Five levels of self-evolution, shown as divers descending from output-level to criterion-level changes
Figure 1. Changes at L0 are limited to the current task, while those at L1, L2, L3, and L4 can carry over to future tasks or updates. Reliable self-evolution requires evidence that remains outside the relevant update boundary.
◎ Overview

A structural map of reliable self-evolution

The survey asks two organizing questions: what changes, and what evidence can support improvement?

Organization of the survey across framing, five self-evolution levels, reliability analysis, and outlook
Figure 2. Part I frames self-evolution and RSI, Part II maps methods from L0 to L4 by evolution depth, and Part III analyzes reliability and open problems.

Classification by the deepest active evolution target

Each transition is classified by the deepest evolution target whose active semantic change affects a decision-relevant output, update, or judgment—not by its algorithm name, training stage, or runtime components.

L0 is task-local; L1–L4 require a retained change that affects later independent tasks or future updates. The levels describe how far a change reaches, not how capable or reliable the system is.

01

Evolution target

Classify the transition by the deepest object that actively changes.

02

Persistence boundary

Separate task-local revision from retained changes that affect future tasks or updates.

03

RSI frontier

Under the structural definition, recursive self-improvement begins at L3 and extends at L4.

04

Reliability evidence

Keep evaluation and oversight independent of the update and matched to the claim’s scope.

1 Foundations

The self-evolution loop needs an external gate

Execution creates experience; an improver proposes change; an independent audit decides whether that change may persist.

The self-evolution loop with task execution, change proposal, external audit, rollback, acceptance, and human escalation
Figure 3. The evidence source and acceptance gate stay outside the update boundary, so the loop cannot rewrite them.

01 Execute & observe

Run a task and collect outputs, trajectories, tool results, and experience.

02 Propose & select

The improver proposes candidate changes and the criterion judges among them.

03 Audit & control

External evidence supports accept, reject and roll back, or escalate to a human.

2 Taxonomy

Five nested levels of self-evolution

Deeper levels retain control over progressively more consequential parts of the update process.

L0

Output-Level

Current output or task-local trajectory · Self-confirmation

L1

Model-Level

Trainable model or policy state · Model collapse

L2

Scaffold-Level

Prompts, skills, memory, workflows, and harness · Scaffold overfitting

L3

Improver-Level

Procedure governing future updates · Metric capture

L4

Criterion-Level

Evaluation protocols, rewards, constraints, or values · Criterion drift

Retention boundary: L0 ends with the current task. RSI frontier: L3 begins self-modification of the improvement mechanism.

L0 Output-Level Self-Evolution

Revise the current output; keep the agent fixed

Deepest active target: current output or task-local trajectory. Characteristic failure: self-confirmation.

42 cataloged works
Task-local workflow of output-level self-evolution through reflection, exploration, and checking
Figure 4. Reflection, exploration, and verification revise the current output while the underlying agent setup stays fixed, so independent tasks start fresh.

Task-Local Boundary & Persistence 1

Changes remain inside the current task and do not persist into later independent tasks.

Iterative Revision 11

Self-feedback, repair, role feedback, and tool results revise a draft in place.

Search, Verification & Acceptance 28

Sample paths, compare candidates, verify with sources or tests, then accept or reject.

Reliability & Persistence Limit 2

Self-repair can fail when the same system both generates and confirms its answer.

L1 Model-Level Self-Evolution

Retain changes in the trainable model or policy

Deepest active target: trainable model or policy state. Characteristic failure: model collapse.

137 cataloged works
Single-model self-training, competitive self-play, and cooperative co-evolution
Figure 5. The three training relations, read left to right as the party emitting the training signal moves further from the trainee and the signal becomes harder to fabricate.

Single-Model Self-Training 51

Select existing outputs or manufacture synthetic supervision for retained policy updates.

Competitive Self-Play 41

Competing copies or specialized proposer–solver loops create adversarial learning signals.

Cooperative Co-Evolution 34

Teachers, critics, co-trainees, or populations improve together and share signal.

Fixed-Scaffold Reliability Limit 11

Guard against model collapse, forgetting, tail narrowing, and weak generalization.

L2 Scaffold-Level Self-Evolution

Evolve the structures that surround the model

Deepest active target: the scaffold. Characteristic failure: scaffold overfitting.

257 cataloged works
The widening scope of scaffold-level self-evolution from prompts to runtime harnesses
Figure 6. The scaffold widens from a prompt or code artifact to the runtime harness that encloses them all, while the improver and criterion stay fixed.

Definition & Boundary 4

Separate retained scaffold changes from model updates and improver-level change.

Prompts & Programs 17

Rewrite language control surfaces or executable artifacts.

Architecture & Workflows 77

Search structures, roles, communication, routing, and multi-agent workflows.

Skills & Experience 106

Form, verify, curate, share, and deploy reusable skills.

Memory & Retrieval 31

Evolve write, recall, structure, provenance, and deletion policies.

Runtime Harness & Reliability 22

Retain editable loops while diagnosing drift and preserving rollback.

L3 Improver-Level Self-Evolution

Change how future updates are proposed and managed

Deepest active target: the improver. Characteristic failure: metric capture.

21 cataloged works
An improver modifying its own improvement mechanism
Figure 7. A candidate successor to the current improver is externally audited and, if promoted, governs later proposal, selection, commit, and rollback under a fixed criterion.

Self-Referential Agents 10

Agents rewrite or replace the mechanism that performs their future improvement.

Learning Better Improvement Strategies 7

Use experience to discover stronger update, optimization, or training strategies.

Fixed-Criterion Reliability Limit 4

Improvement can become metric capture when the updater exploits a fixed criterion.

L4 Criterion-Level Self-Evolution

Evolve how present and future updates are judged

Deepest active target: the criterion. Characteristic failure: criterion drift.

30 cataloged works

Evolving Evaluation Mechanisms 11

Develop rubrics, evaluators, meta-judges, and co-evolving graders.

Evolving Tasks & Objectives 19

Change the tasks, environments, objectives, or feedback used to judge improvement.

Reliability boundary

Use protected outcomes, cross-criterion comparison, external authorization, and constraints.

3 Cross-Level Reliability

Match every evolution target with an external audit

The rise of the ladder is self-evolution depth, not capability or reliability.

Reliability ladder pairing each self-evolution level with an external audit and control
Figure 8. Each step is a deeper evolution target, and each card pairs it with the external audit and control that a claim at that depth requires.
01 / EVIDENCE

External audit across levels

Use evidence outside the relevant update boundary and matched to the improvement claim.

02 / HORIZON

Level-specific failure modes

Extend evaluation from a current output to transfer, retention, descendants, and protected outcomes.

03 / INDEPENDENCE

Preserve audit independence

Protect evidence sources, acceptance gates, lineage, rollback, and human escalation from compromise.

4 Applications & Outlook

From updates to staged deployment

Wider deployment requires staged evaluation against a declared external target.

Applications of self-evolving agents across engineering, persistent digital agents, scientific discovery, and high-stakes systems
Figure 9. Across executable engineering, persistent digital agents, scientific discovery, and embodied or high-stakes systems, evidence ranges from executable checks to expert review and backtests.
EE

Executable Engineering & AI Development

Build, test, and optimize software, algorithms, infrastructure, and data workflows.

PA

Persistent Digital Agents

Adapt across sessions in live environments while reusing workflows and learning from failures.

SD

Scientific Discovery

Accelerate analysis, computing, experimentation, and research with verifiable outputs.

HS

Embodied & High-Stakes Systems

Adapt in robotics, healthcare, finance, education, power, and law with consequential outcomes.

5 Reading List

Browse 549 manuscript-used works

Every paper appears once in its primary section and follows the canonical L0–L4 catalog.

24 Surveys & Positioning Field positioning and adjacent literature ↗ 42 L0 · Output-Level Task-local revision and verification ↗ 137 L1 · Model-Level Self-training, self-play, and co-evolution ↗ 257 L2 · Scaffold-Level Programs, workflows, skills, memory, harness ↗ 21 L3 · Improver-Level Self-reference and learned improvement ↗ 30 L4 · Criterion-Level Evaluation mechanisms, tasks, and objectives ↗ 8 Cross-Level Reliability Evidence, acceptance, control, independence ↗ 30 Open Problems & Outlook Evolution, evaluation, applications, governance ↗
Open the complete paper catalog

The repository README remains the single source of truth for paper records and official links.

” About

Cite, contribute, and keep the catalog reliable

Contributions should first be cited by the survey, then placed once in their primary taxonomy section.

Contributing

Correct a manuscript-used record, or add the paper to the manuscript before proposing it here. Use a canonical paper URL and include code or project links only when they are official.

Original text and images in the repository are licensed under MIT. Linked papers, repositories, project pages, names, and third-party metadata retain their respective terms.