Evolution

Evolution improves the workspace an agent reads while keeping the base model and benchmark tasks fixed. Prompts and skills change under version control. Each candidate is checked against selected public AEC-Bench tasks before the search policy decides what to keep.

Use the YAML workflow for a standard run. Use the Python API when you want to replace the candidate checks, the candidate proposer, or both without rebuilding the evolution loop.

The cycle

Each cycle has six responsibilities:

Step	Owner	What happens
Select	Search policy	Choose the parent and any reference candidates
Check parent	`CandidateChecks`	Plan a candidate-independent batch and bind its `TrialRecord` evidence to the parent
Propose	Candidate proposer	Work in scratch and return a candidate, abstain, or finish because the budget is exhausted
Check child	`CandidateChecks`	Run the child on the same ordered cases used for the parent
Gate	Search policy	Accept, reject, or skip from exact evidence
Persist	Application shell	Commit accepted material and update lineage, archive, graveyard, and reports

The canonical workspace changes only after acceptance. The outer evolution loop owns scoring, commits, and search state.

The built-in proposer is Agentic Variation (AVO). It can inspect, edit, test, diagnose, and repair several scratch revisions before it submits a candidate.

Setup

Create a workspace:

uv run aec-bench evolve init workspaces/voltage-drop-evo \
  --name voltage-drop-evo \
  --harness rlm

The workspace contains a manifest, system prompt, and skill directory. Treat it like source code: review diffs, keep changes scoped, and keep benchmark answers out of prompts and skills.

Configuration

Evolution is normally driven by YAML:

workspace_path: workspaces/voltage-drop-evo

models:
  classifier: env:AWS_HAIKU_MODEL_ID
  evolver: env:AWS_SONNET_MODEL_ID

solver:
  name: solver-v1
  harness: rlm
  model: env:AWS_SONNET_MODEL_ID

tasks:
  domains: [electrical]
  include_patterns: ["electrical/*"]
  difficulties: ["easy", "medium"]

backend: local
batch_size: 5
max_cycles: 10
improvement_threshold: 0.01
stagnation_window: 5
strategy: hill_climb
timeout: 1800

Run and inspect it with:

uv run aec-bench evolve run --config evolution.yaml --tasks-root tasks
uv run aec-bench evolve history workspaces/voltage-drop-evo

--tasks-root binds the YAML selectors to runnable public tasks. Benchmark-performance claims require task-backed evidence; composition-only runs are useful for integration checks.

Three kinds of checks

The checks use the same task-running machinery, but they run at different points and have different authority.

Check	When it runs	What it can do
Revision checks	Inside AVO, while it edits a scratch candidate	Give the proposer feedback for another edit, restore, abstain, or submit
Selection checks	Outside AVO, after one child is submitted	Compare the parent and child and decide whether the child is accepted
Qualification checks	After evolution is complete	Measure the finished candidate on separate holdout tasks

Think of revision checks as draft feedback and selection checks as the promotion decision:

AVO:
  edit draft -> revision check -> edit again -> revision check
  -> submit child

Evolution:
  selected parent + submitted child -> selection checks
  -> accept or reject

AVO uses revision checks because it is still developing the child. Evolution retains final selection authority and uses selection checks to assess the submitted child against its parent.

Both check types use public tasks. They can share a public batch or use different batches, but their evidence and experiment identities remain separate. Qualification keeps holdout material in a separate post-adaptation evaluation.

Functional composition

Both runtime roles use CandidateChecks, so local composition calls build_local_checks() twice. The first instance belongs to AVO's private revision loop. The second belongs to the outer evolution gate.

from aec_bench.evolution import build_avo, build_local_checks, run_evolution

# Draft feedback: AVO can run these checks after each scratch edit.
revision_checks = build_local_checks(
    task_dirs=task_dirs,
    model=solver_model,
    experiment_id="revision",
    workspace_root=workspace.root,
    candidate_identity=False,  # AVO assigns evidence IDs to each exact revision.
)

propose = build_avo(
    model=proposal_model,
    model_identity=proposal_model_id,
    revision_checks=revision_checks,
    batch_size=config.batch_size,
)

# Promotion decision: Evolution runs these checks on the parent and submitted child.
selection_checks = build_local_checks(
    task_dirs=task_dirs,
    model=solver_model,
    experiment_id="selection",
    workspace_root=workspace.root,
)

result = run_evolution(
    workspace=workspace,
    config=config,
    selection_checks=selection_checks,
    propose=propose,
)

run_evolution() gives the selected parent to propose. The AVO proposer uses revision_checks while it works and returns a CandidateProposal. run_evolution() then uses selection_checks to assess the returned child against its parent. Only selection evidence reaches gate_candidate().

CandidateChecks groups batch planning, execution, and optional observation enrichment for both roles. Separate instances keep draft feedback apart from promotion evidence.

API	Meaning
`CandidateChecks`	Plan and run one consistent set of candidate checks
`build_local_checks()`	Use local AEC-Bench tasks as a check capability
`build_avo()`	Create the built-in candidate proposer
`gate_candidate()`	Return a pure accept, reject, or skip decision
`next_evolution_state()`	Return the next search state without applying effects
`run_evolution()`	Coordinate checks, proposals, decisions, and persistence

Search strategies

strategy: hill_climb selects the best evaluated candidate as the next parent. A valid child can continue until the configured stagnation window is reached. Clearing the improvement threshold resets the stagnation counter and updates the best candidate.

strategy: qd uses a quality-diversity archive. A valid candidate can enter a new archive cell or improve an occupied cell without becoming the global best. The host owns the shortlist, mutation strategy, archive insertion, and strategy-bandit feedback.

Use Swarm when several independent agents should explore this archive in parallel.

Candidate identity and evidence

Each candidate has one stable candidate_id. Its source_revision is the full Git commit containing its source. parent_candidate_id records explicit evolutionary lineage. An optional immutable label provides a human-friendly discovery name.

Parent and child evidence remain separate. Both must use the same ordered evaluation cases before comparison. The task verifier remains the reward authority, and TrialRecord remains the evidence authority.

Rollback accepts a candidate ID or immutable label. It restores that candidate's exact source as a new Git commit and preserves the original candidate.

uv run aec-bench evolve rollback workspaces/voltage-drop-evo <candidate-id>

When to use it

Evolution is useful when:

failures repeat across a meaningful public task set;
the agent workspace has editable prompts and skills;
you have enough tasks and budget to distinguish a real signal from one lucky run; and
a person can review promoted changes before broader use.

For a benchmark run without evolution, use a fixed harness. For multi-step work inside a proposal, use AVO. For parallel quality-diversity search, use Swarm.

On this page