aec-benchaec-bench

Advisor

Use an advisor when a cheaper agent can carry out work but sometimes needs help choosing the next step. The harness sends a focused question to another model, receives one suggestion, and then continues the same trial. The agent remains responsible for tools, files, and action selection.

This page covers advice inside a normal agent-harness trial. Evolution's AVO advisor is separate: it helps revise a workspace proposal after repeated failed checks.

How it works

Inside a harness run, an adapter can issue an AdvisorRequest whenever it hits a decision point:

@dataclass(frozen=True)
class AdvisorRequest:
    goal: str                # what the agent is trying to do overall
    problem: str             # the specific thing it is stuck on now
    attempt: str | None = None

The advisor returns a structured response:

@dataclass(frozen=True)
class AdvisorResponse:
    advice: str                 # strategic guidance
    suggested_action: str       # one concrete next step
    confidence: float           # 0.0 to 1.0
    reasoning: str              # why this suggestion

The advisor returns guidance; the agent decides whether to follow it.

Configuration

Advisor settings belong to the harness config:

[advisor]
model = "au.anthropic.claude-sonnet-4-6"  # the advising model
max_uses = 5                              # calls allowed per trial
max_response_tokens = 500                 # keep advice concise
context_window = 10                       # recent turns to include
enabled = true

max_uses is a hard cap per trial.

What the advisor sees

Different agent harnesses provide different context. A tool loop might show recent tool calls and results. An RLM harness might show recent REPL commands and scratchpad notes. The internal AdvisorContextStrategy protocol builds this context:

class AdvisorContextStrategy(Protocol):
    def build_advisor_context(
        self,
        request: AdvisorRequest,
        conversation_state: Any,
    ) -> list[dict[str, str]]: ...

This lets the same advisor model help different harnesses without receiving their full transcripts.

Usage tracking

Advisor calls are tracked separately from the main agent's token budget:

class AdvisorUsageStats(StrictModel):
    calls_made: int
    calls_remaining: int
    advisor_input_tokens: int
    advisor_output_tokens: int
    advisor_cost_usd: float

Reports can surface these alongside the base cost so readers can see the split: "this run cost USD 0.42 to execute, plus USD 0.18 in advisor calls".

Failure handling

If the advisor call fails (network error, provider timeout, parsing failure), the agent receives a safe fallback response:

AdvisorResponse(
    advice="Advisor unavailable - proceed on your own judgement",
    suggested_action="continue",
    confidence=0.0,
    reasoning="advisor call failed",
)

A zero-confidence fallback lets the trial continue under the agent's own judgement.

When to use it

The advisor can help when the base model is fast and cheap but often chooses the wrong direction. Two useful signals are:

  • The agent makes early choices that repeatedly derail the rest of the trial.
  • The agent can execute a good plan once it receives one.

Compare advisor cost and task success against a longer turn budget or a stronger base model. Use the option that performs better for the task at an acceptable cost.

On this page