aec-benchaec-bench

Execution Control

Execution control runs the exact trials in a ready RunPlan. It gives each planned trial one TrialWorkItem, records each execution try as an Attempt, and publishes the selected result as a TrialRecord.

RunPlan
   ↓ create
TrialWorkItem
   ↓ lease
Attempt → BackendSubmission
   ↓ collect
AttemptReceipt
   ↓ finalise
TrialFinalization → TrialRecord

Operational state

OperationalStore uses SQLite to coordinate a run's planned trials and execution attempts. Short transactions update scheduler state while portable plans and trial evidence stay in the evidence store.

The store records the state needed to dispatch work and recover a process:

RecordRole
TrialWorkItemA planned trial with its execution routes, resource requirements, priority, and retry policy
AttemptOne execution try under a stable trial and work-item identity
LeaseA time-limited worker claim with heartbeat and release state
BackendSubmissionThe provider submission identity and observed backend state
AttemptReceiptVersioned collected facts, output references, verifier receipt, usage, and reconciliation state
TrialFinalizationThe versioned pointer to the published final trial record

These contracts use UUIDv7 identities and timezone-aware timestamps. A retry creates a new attempt under the same planned trial. A finalization binds the published TrialRecord to its selected attempt.

Scheduler policy

ExecutionPolicy fixes the scheduling rules as part of the resolved run condition:

ControlScope
Global concurrencyMaximum concurrent work across the scheduler
Scoped concurrencyLimits by run, backend, provider route, model route, resource class, or execution family
Retry policyWhich failures can be retried and when
Lease timingHow long a worker can claim work and when it must renew ownership
Priority ageingHow waiting affects scheduling priority

The local scheduler leases ready work when all applicable limits allow it. It applies priority ageing, records backoff, renews heartbeats, and releases or expires leases. Retry policy classifies infrastructure failures separately from benchmark and invalidating failures. Unknown external state enters reconciliation before another attempt can be selected.

The CLI planning path writes ExecutionPolicy(max_concurrency=1). The policy contract is available to Python composition and persisted plan inspection.

Start, resume, and cancel

Start or resume a persisted local artefact plan with explicit roots:

uv run aec-bench run start <run-id> \
  --operational-store artefacts/operational.sqlite \
  --plan-root artefacts/runs \
  --tasks-root tasks

uv run aec-bench run resume <run-id> \
  --operational-store artefacts/operational.sqlite \
  --plan-root artefacts/runs \
  --tasks-root tasks

run start accepts a ready persisted plan and executes its local artefact trials. run resume loads the same plan, expires old leases, reconciles uncertain work, and continues ready work. Both commands validate the saved task snapshot and local backend before execution.

Request cancellation with the operational store:

uv run aec-bench run cancel <run-id> \
  --operational-store artefacts/operational.sqlite

Queued work is cancelled. Active work records a cancellation request and the backend result is reconciled before the work reaches a terminal state.

Progress

Use RunProgress to see how much of the plan has finished and what is holding up the rest. It combines the authoritative RunPlan with operational rows:

Progress detailCounts and values
PlanPlanned and remaining work
Active workQueued, leased, and running work items and trials
Terminal outcomesSucceeded, failed, cancelled, and invalid work items and trials
GapsUnknown and missing work items and trials
RecoveryAttempts, backend submissions, retries, and active or expired leases
TimingRecorded timestamps

The CLI reads this projection with:

uv run aec-bench run status <run-id> \
  --operational-store artefacts/operational.sqlite \
  --plan-root artefacts/runs

The TUI exposes the same projection in its run-progress view. The web API returns it from GET /api/runs/{run_id}/status when the web process has an operational store and plan root configured.

Execution-family composition

The scheduler accepts each execution family's worker through a Python adapter. run start and run resume use the local artefact adapter; Interactive World, finite lifecycle, and Harbor integrations compose their own workers against the same operational contracts.

Each family keeps its task meaning, state transitions, verifier, and evidence rules. The scheduler controls work ownership and recovery, while the task family defines result meaning.

See Runs and Plans for plan identity and Contracts for the validated data shapes.

On this page