Agentic Variation (AVO)

Agentic Variation (AVO) is the part of Evolution that drafts changes to an agent workspace, including its system prompt and skills. It can inspect results, edit a copy of the workspace, test the changes, diagnose failures, and revise the draft before submission.

The design is based on Chen et al., “AVO: Agentic Variation Operators for Autonomous Evolutionary Search”. In AEC-Bench, AVO proposes a candidate; Evolution decides whether to keep it.

Where AVO fits

At the start of a cycle, Evolution chooses:

the parent workspace to improve;
the goal and editing strategy; and
any reference candidates that may contain useful approaches.

A reference candidate is another saved workspace version chosen as a read-only example. The API calls these candidates inspirations. The selected parent remains the sole parent, and AVO chooses which ideas to reproduce in its draft.

AVO works on a private copy of the parent and sees only the reference candidates supplied by Evolution. After AVO submits a child, Evolution runs selection checks and decides whether to accept it. The canonical workspace changes only after acceptance.

See Three kinds of checks for the difference between revision checks, selection checks, and holdout qualification.

Proposal flow

Before AVO starts, Evolution supplies the parent and its results, the goal, the editing strategy, selected reference candidates, and relevant history from earlier cycles. The Python API carries these inputs in a CandidateProposalRequest.

AVO then:

chooses a fixed set of public tasks for draft feedback;
runs those tasks on the parent;
creates a private copy of the workspace;
lets the agent edit, check, diagnose, and restore drafts; and
returns the latest checked draft or finishes without a candidate.

The proposal status is submitted, abstained, budget_exhausted, or cancelled. Only submitted can contain child material. run_evolution() still runs selection checks before it can persist the child.

Tool surface

The agent receives these typed tools:

Tool	Purpose
`inspect_parent_results`	Read the parent results
`inspect_current_candidate`	Read current scratch material
`inspect_inspirations`	Read the reference candidates selected for this proposal
`inspect_previous_cycles`	Read earlier evolution cycles
`inspect_rejected_candidates`	Read rejected-candidate summaries
`read_program_guidance`	Read the workspace guidance
`edit_candidate`	Apply validated prompt or skill edits in scratch
`test_candidate`	Run the fixed revision checks for the current revision
`restore_candidate`	Restore a previously checked draft
`submit_candidate`	Submit the current checked draft
`abstain`	Finish without a child
`request_advice`	Ask for a new direction when the current approach is exhausted

edit_candidate changes only the private copy. Each new edit makes the previous revision results stale.

How revision checks stay comparable

AVO uses the same ordered public tasks for every revision in a proposal. This lets it compare a new draft with the parent and earlier drafts on the same work.

Every check is tied to the workspace content that was tested. A RevisionAttempt records the workspace snapshot, the reason for the edit, its TrialRecord results, and the resources used. Submission requires results from the exact revision being proposed.

Revision checks use public development tasks. Holdout tasks remain reserved for qualification after adaptation.

Limits

An AVOBudget uses these defaults:

Limit	Default
Model requests	12
Tool calls	40
Revision checks	7
Elapsed time	1,800 seconds
Consecutive evaluation errors	2
Stagnant revision checks	3
Advisor interventions	0

AVOBudget() by itself allows no advisor calls. The normal build_avo() setup enables one unless you supply a different budget.

Token and USD limits are optional. If a configured limit needs a token count or cost that the provider did not report, AVO stops. A 0.0 cost means known free usage; None means unknown.

Conditional advice

If AVO gets stuck, a second model can suggest another direction. Advice can run:

after three valid checks without progress;
after two consecutive invalid or failed checks; or
when the main model calls request_advice.

The advisor sees short summaries of the goal, attempts, and remaining budget. Its role is to suggest one to three directions or confirm a validation failure.

This advisor is separate from the trial-level Advisor tool.

Memory and resume

AVO keeps at most 24 short facts between revisions and proposal cycles. This compact memory prioritises the best result, recent failure types, and useful editing directions.

When checkpointing is enabled, AVO saves enough state to continue an interrupted proposal. This includes:

the parent and current scratch workspace;
completed revision checks and their evidence;
budget and usage totals;
retained memory and advice; and
whether a model call or check was interrupted.

On resume, AVO verifies that the run, parent, workspace, and configuration still match. If the interruption occurred during a model call or revision check, AVO determines whether that work completed before retrying it. Completed paid work is not repeated. If the proposal already finished, resume returns the recorded result.

Python composition

build_avo() returns the proposer expected by run_evolution(). The revision_checks input is the public-task draft feedback capability described on the Evolution page.

from aec_bench.evolution import build_avo

propose = build_avo(
    model=proposal_model,
    model_identity=proposal_model_id,
    revision_checks=revision_checks,
    batch_size=5,
    advisor_model=advisor_model,              # optional
    advisor_model_identity=advisor_model_id,  # optional
)

When no advisor model is supplied, AVO reuses the proposal model. The aec-bench evolve run command builds the revision checks and proposer from YAML, so Python composition is optional.

What is tested

The AEC-Bench test suite covers AVO tools, budgets, revision evidence, advice, memory, and resume behavior with provider-free test doubles. Evidence that a particular model improves a workspace requires a separate model-backed study.

Measure model quality and cost with a public-task development run, then use separate holdout qualification before promoting the result.