Prime Agent

Prime Agent is a local external-process agent harness. aec-bench launches the upstream executable and adapts its output into the same verification, evaluation, and evidence boundaries used by other harnesses.

Prime Agent executes one local actor process. Prime Lab exports environments for hosted evaluation and training.

Install

The artefact-task adapter is in the provider-free base package. Install the upstream executable separately:

pip install aec-bench
# Follow the upstream Prime Agent installation so prime-agent is on PATH.

The executable is maintained by Prime Intellect. aec-bench checks for it when the harness is selected and reports a direct installation error if it is absent.

The integration is tested against Prime Agent 0.7.0 and JSON event-stream version 3. Unsupported stream versions fail explicitly.

Artefact tasks

Use the public harness name with run-local:

aec-bench run-local tasks/<task> \
  --harness prime-agent \
  --model anthropic/<model-id>

aec-bench stages the task, starts Prime Agent in JSON mode, and then uses the normal output, normalisation, verifier, evaluation, and ledger-import flow. The model string is passed to Prime Agent unchanged.

Each trial uses isolated configuration and session directories under logs/prime/. Ambient skills, extensions, prompt templates, themes, and context files are disabled. Provider credentials can still be inherited by the process; provenance excludes them.

The retained evidence includes:

• the Prime JSON event stream;
• standard error output;
• a run summary;
• saved Prime session files;
• the submitted task output; and
• normal trial, evaluation, usage, and artefact references.

Unsupported event streams fail explicitly and retain their raw execution evidence.

Interactive World sessions

Install the ACP client support for Interactive Worlds and lifecycles:

pip install "aec-bench[prime-agent]"

The ACP integration connects Prime Agent to selected Interactive Worlds. One generic actor endpoint exposes only the installed world actions and actor-visible results. Task state, host controls, verifier data, and world persistence remain host-owned.

AEC-Bench currently supports:

• one wastewater pump-station stewardship session;
• one dam-seepage monitoring episode; and
• one fresh Prime session per checkpoint for the hydraulic-review lifecycle.

Dam and pump sessions use the shared run_prime_world_actor_session() actor boundary. A complete benchmark call uses run_dam_seepage_trial() or run_pump_station_trial() instead. These functions run the task-owned journey, verify and evaluate the result, retain the evidence artefacts, and return one TrialRecord.

Use asynchronous run_world_experiment() to apply a complete world trial function to an ordered plan. Each trial covers the full world journey and can contain several actions or provider sessions. Returned TrialRecord artefacts remain available after temporary session cleanup.

The pump trial receives a world request selected by the host. The dam trial receives the registered dam profile selected by the host. After the actor process closes, the task owner replays the accepted actions and runs verification and evaluation outside Prime Agent.

Task termination, host truncation, model completion, and provider error remain separate facts.

Skills and isolation

World sessions install explicit generic skills by default. Prime receives the task objective, actor-visible starting material, and the allowed actor capability. A study can add task-family methods or solution plans as a declared treatment.

The pump-station journey can close Prime at an Operations boundary, apply one deterministic task-owned host control, and start a fresh session from the exact new snapshot. The host chooses and applies that control. Dam and pump state, journeys, controls, persistence, verification, and evaluation remain separate and task-owned.

Prime runtime state is isolated for each session. World state, replay, and verification remain with the task owner.

Security boundary

For artefact tasks, Prime Agent executes model-generated Python and shell commands with the current user's operating-system permissions. Trial-specific directories and disabled ambient configuration improve reproducibility, but they are not a security sandbox.

ACP world and lifecycle sessions select an explicit isolation mode. development_same_user has the same local security limit. On macOS, macos_sandbox uses sandbox-exec and a scoped actor socket; only a successful session in this mode is marked as benchmark-valid isolation.

Use unsandboxed local paths only with trusted task packages.

Evidence authority

Prime event logs are execution evidence. Task verifiers supply task-owned evidence, and evaluation owns reward and validity. RunManifest records shared run identity, while each TrialRecord records one reportable trial result.

See Agent Harnesses for comparison with the other local strategies and Contracts for the evidence model.