Swarm

aec-bench swarm runs several evolution agents in parallel. Each agent changes a private copy of the workspace. A shared archive keeps strong candidates that solve the tasks in different ways. This approach is called quality-diversity search.

When to use it

Use swarm when:

• A single evolution run keeps converging on one local strategy.
• You want diversity across cost, verification depth, tool use, exploration, deliberation, and reward.
• You have enough budget for parallel evaluation.
• You can review archive entries before promoting a candidate.

Use Evolution for one search loop. Each swarm worker uses its own AVO candidate proposer, scratch workspace, cancellation signal, bounded memory, and checkpoint identity.

Setup

Start with a normal evolution workspace:

uv run aec-bench evolve init workspaces/my-swarm \
  --name "My Swarm Experiment" \
  --harness rlm

Then create a swarm.yaml:

task:
  workspace: ./workspaces/my-swarm
  task_path: tasks/electrical/voltage-drop

agents:
  count: 4
  default_model: au.anthropic.claude-sonnet-4-6

budget:
  max_cost_usd: 20.0
  eval_budget_usd: 5.0
  wind_down_threshold: 0.8
  final_threshold: 0.95

evaluation:
  timeout: 300
  backend: local

evolution:
  batch_size: 1
  improvement_threshold: 0.01

heartbeat:
  pivot_after: 5

Run

uv run aec-bench swarm run swarm.yaml

swarm run is a foreground command. Keep its process attached until the agents finish or the recorded spend reaches the configured maximum.

Swarm state is written under the workspace's _swarm_runs/ directory:

_swarm_runs/
├── events.jsonl
├── swarm_state.json
├── candidates.json
├── archive.json
├── graveyard.json
├── lineage.json
├── notes.json
├── budget.json
└── summary.json

swarm_state.json records the current decisions. candidates.json stores the exact candidate content. The other files store accepted candidates, rejected candidates, ancestry, notes, budget, and a readable event history. Recovery uses the state and candidate files; the event history explains what happened.

The current output layout uses one flat state directory. A later run in the same workspace uses the same snapshot paths and appends to the same event log. Use a separate workspace when runs must remain isolated.

Status and current control limits

The current CLI provides run, status, and history:

uv run aec-bench swarm status <run-id> \
  --state-dir workspaces/my-swarm/_swarm_runs

uv run aec-bench swarm history \
  --state-dir workspaces/my-swarm/_swarm_runs

status validates swarm_state.json and its supporting candidate, archive, graveyard, lineage, note, and budget material. It rejects a different run ID or unresolved candidate identity. history scans validated persisted swarm states.

swarm run is still a foreground process. The CLI has no stop or resume command. Keep the process attached until completion, and use a separate workspace when runs must remain isolated.

How coordination works

Each agent receives a parent workspace, selected reference candidates, an editing strategy, and a goal. It runs AVO in private and returns a proposal plus its usage. The manager owns scoring and archive decisions.

In the Python API, these values are SwarmAssignment and CandidateProposal.

The manager then:

1. plans a shared selection batch;
2. checks the assigned parent and submitted child without blocking other evaluations;
3. attaches the resulting TrialRecord evidence to the exact candidates;
4. updates selection, rejection, budget, and ancestry state in one short protected operation; and
5. saves the shared state and candidate content.

Evaluations can run at the same time. SwarmState records the accepted decisions; the event log and agent reports provide supporting information.

Behaviour descriptors

The archive groups candidates by behaviour as well as reward:

Two workspaces can have the same reward and still be worth keeping if they solve the task with different behaviours.

Budget behaviour

The budget pool is shared across agents. At the wind-down threshold, agents receive a warning to focus their remaining evaluations. At the final threshold, they receive a stronger warning. Agents retire only after recorded spend reaches the configured maximum, so concurrently running evaluations can take total spend past that value.

Start small:

Costs depend heavily on adapter, model, task length, and verification behaviour.

Pivot heartbeat

If an agent records too many non-improving evaluations, the manager injects a pivot instruction to direct the remaining budget towards materially different strategies.

Use pivoting as a guardrail. Normal inspection and evaluation establish whether an archive entry is a benchmark-relevant improvement.