Prime Agent

Prime Agent is a local external-process agent harness. aec-bench launches the upstream executable and adapts its output into the same verification, evaluation, and evidence boundaries used by other harnesses.

Prime Agent executes one local actor process. Prime Lab exports environments for hosted evaluation and training.

Install

The artefact-task adapter is in the provider-free base package. Install the upstream executable separately:

pip install aec-bench
# Follow the upstream Prime Agent installation so prime-agent is on PATH.

The executable is maintained by Prime Intellect. Install it on PATH before selecting this harness.

The integration is tested against Prime Agent 0.7.0 and JSON event-stream version 3.

Artefact tasks

Use the public harness name with run-local:

aec-bench run-local tasks/<task> \
  --harness prime-agent \
  --model anthropic/<model-id>

aec-bench stages the task and starts Prime Agent in JSON mode with the supplied model string. After execution, the normal trial pipeline normalises and verifies the output before evaluating and importing the result.

Each trial uses isolated configuration and session directories under logs/prime/. Ambient skills, extensions, prompt templates, themes, and context files are disabled. Provider credentials can still be inherited by the process; provenance excludes them.

The retained evidence includes:

• the Prime JSON event stream;
• standard error output;
• a run summary;
• saved Prime session files;
• the submitted task output; and
• normal trial, evaluation, usage, and artefact references.

Child traces

After a JSON run or ACP session closes, AEC-Bench converts the saved session evidence into trajectory.jsonl. The trace retains parent and child sessions, their messages, and recorded usage. Each child can link back to the tool call that started it, including calls that finish out of order.

Use these links to follow a delegated request from the parent tool call to the child's work and response. The entries use the shared trajectory schema.

Interactive World sessions

Install the ACP client support for Interactive Worlds and lifecycles:

pip install "aec-bench[prime-agent]"

The ACP integration connects Prime Agent to selected Interactive Worlds. One generic actor endpoint exposes only the installed world actions and actor-visible results. Task state, host controls, verifier data, and world persistence remain host-owned.

AEC-Bench currently supports:

• one wastewater pump-station stewardship session;
• one dam-seepage monitoring episode; and
• one fresh Prime session per checkpoint for the hydraulic-review lifecycle.

Dam and pump sessions use the shared run_prime_world_actor_session() actor boundary. For a complete benchmark trial, call run_dam_seepage_trial() or run_pump_station_trial(). These functions run the task-owned journey, verify and evaluate the result, retain the evidence artefacts, and return one TrialRecord.

Use asynchronous run_world_experiment() to apply a complete world trial function to an ordered plan. Each trial covers the full world journey and can contain several actions or provider sessions. Returned TrialRecord artefacts remain available after temporary session cleanup.

The pump trial receives a world request selected by the host. The dam trial receives the registered dam profile selected by the host. After the actor process closes, the task owner replays the accepted actions and runs verification and evaluation outside Prime Agent.

Task termination, host truncation, model completion, and provider error remain separate facts.

Skills and isolation

World sessions install explicit generic skills by default. Prime receives the task objective, actor-visible starting material, and the allowed actor capability. A study can add task-family methods or solution plans as a declared treatment.

At a pump-station Operations boundary, the host can close Prime and apply a deterministic control before starting a fresh session from the resulting snapshot. Each world owns these state changes and the rules for retaining and evaluating its evidence.

Prime runtime state is isolated for each session. World state, replay, and verification remain with the task owner.

Security boundary

For artefact tasks, Prime Agent executes model-generated Python and shell commands with the current user's operating-system permissions. Trial-specific directories and disabled ambient configuration improve reproducibility.

ACP world and lifecycle sessions select an explicit isolation mode. development_same_user has the same local security limit. On macOS, macos_sandbox uses sandbox-exec and a scoped actor socket; only a successful session in this mode is marked as benchmark-valid isolation.

Use unsandboxed local paths only with trusted task packages.

Evidence authority

Prime event logs are execution evidence. Task verifiers supply task-owned evidence, and evaluation owns reward and validity. RunManifest records shared run identity, while each TrialRecord records one reportable trial result.

See Agent Harnesses for comparison with the other local strategies and Contracts for the evidence model.