LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Valve Failure Safe State Repair Package

Calculates source-bound valve failure repair metrics from a deterministic SSC-18 task-owned source pack. The template combines failed-signal threshold, fail-safe valve capacity, bypass capacity, safe-flow and runtime margins, source resolution, conflict closure, and an overall synthetic pass score.

with-tool: The model is given an executable Python calculator script.

Standards

ISA-5control narrative workflowfailure mode table workflow

Tags

deterministicelectricalfailureinstrumentationrepairsafe-statevalve

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

14 inputs

Always given

Included directly in every task prompt.

14
Show 14 inputs

  • Failed signal

    failed_signal_ma

    LOOP-18-FAIL-07 failed signal value

    3.6 mA

  • Fail threshold

    fail_threshold_ma

    MODE-18-FAIL-07 fail-detect threshold

    3.8 mA

  • Fail closed cv

    fail_closed_cv

    VDS-18-FAIL-07 fail-closed available Cv

    5

  • Required safe cv

    required_safe_cv

    NARR-18-SAFE-07 required safe-state Cv

    4

  • Bypass cv

    bypass_cv

    VDS-18-FAIL-07 bypass Cv

    12

  • Required bypass cv

    required_bypass_cv

    NARR-18-SAFE-07 required bypass Cv

    10

  • Safe flow

    safe_flow_m3_h

    NARR-18-SAFE-07 safe-state flow

    15 m3/h

  • Minimum safe flow

    minimum_safe_flow_m3_h

    MODE-18-FAIL-07 minimum safe flow

    12 m3/h

  • Tank volume

    tank_volume_m3

    PID-18-FAIL-07 available tank volume

    100 m3

  • Drawdown flow

    drawdown_flow_m3_h

    MODE-18-FAIL-07 safe-state drawdown flow

    15 m3/h

  • Required safe duration

    required_safe_duration_h

    NARR-18-SAFE-07 required safe duration

    6 h

  • Source items resolved

    source_items_resolved

    RESPONSE-18-FAIL-07 resolved source item count

    6

  • Source items total

    source_items_total

    RESPONSE-18-FAIL-07 total source item count

    6

  • Unresolved conflict count

    unresolved_conflict_count

    RESPONSE-18-FAIL-07 unresolved conflict count

    0

Scored outputs

9 outputs

Failed signal margin ma

failed_signal_margin_ma

Fail-detect threshold minus failed signal

Scores if within ±3% of the reference value.

Fail closed cv margin

fail_closed_cv_margin

Fail-safe Cv minus required safe Cv

Scores if within ±3% of the reference value.

Bypass cv margin

bypass_cv_margin

Bypass Cv minus required bypass Cv

Scores if within ±3% of the reference value.

Safe flow margin

safe_flow_margin_m3_h

Safe flow minus minimum safe flow

Scores if within ±3% of the reference value.

Safe runtime h

safe_runtime_h

Safe-state runtime from available tank volume

Scores if within ±3% of the reference value.

Safe runtime margin h

safe_runtime_margin_h

Safe runtime minus required duration

Scores if within ±3% of the reference value.

Source resolution fraction

source_resolution_fraction

Resolved source items divided by total source items

Scores if within ±3% of the reference value.

Unresolved conflict count

unresolved_conflict_count

Unresolved source conflict count

Scores if within ±3% of the reference value.

Overall pass score

overall_pass_score

Pass score for valve failure safe-state repair checks

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-18 valve failure repair package
medium:
All source-pack values given for the SSC-18 valve failure repair package
hard:
All source-pack values given for the SSC-18 valve failure repair package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • valve-failure-safe-state-repair-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are checking `SSC-18-LH-07`, a task-owned synthetic valve failure and safe-state repair package.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. P&ID, valve datasheet, loop schedule, failure-mode table, and control narrative workflows shape the context only; this instance does not parse real P&IDs, PLC logic, trip matrices, or authority-approved repair records.4 5Source pack:6 7- P&ID: `PID-18-FAIL-07`8- Valve datasheet: `VDS-18-FAIL-07`9- Loop schedule: `LOOP-18-FAIL-07`10- Failure mode table: `MODE-18-FAIL-07`11- Control narrative: `NARR-18-SAFE-07`12- Repair response: `RESPONSE-18-FAIL-07`13 14Given values:15 16| Field | Value |17| --- | ---: |18| Failed signal | {{ failed_signal_ma }} mA |19| Fail threshold | {{ fail_threshold_ma }} mA |20| Fail-closed Cv / required safe Cv | {{ fail_closed_cv }} / {{ required_safe_cv }} |21| Bypass Cv / required bypass Cv | {{ bypass_cv }} / {{ required_bypass_cv }} |22| Safe flow / minimum safe flow | {{ safe_flow_m3_h }} / {{ minimum_safe_flow_m3_h }} m3/h |23| Tank volume / drawdown flow | {{ tank_volume_m3 }} m3 / {{ drawdown_flow_m3_h }} m3/h |24| Required safe duration | {{ required_safe_duration_h }} h |25| Source items resolved/total | {{ source_items_resolved }} / {{ source_items_total }} |26| Unresolved conflicts | {{ unresolved_conflict_count }} |27 28Required calculations:29 30- Failed signal margin equals fail threshold minus failed signal.31- Cv margins equal available Cv minus required Cv.32- Safe flow margin equals safe flow minus minimum safe flow.33- Safe runtime equals tank volume divided by drawdown flow.34- Source resolution fraction equals resolved source items divided by total source items.35 36Return one JSON object with keys:37 38```json39{40 "failed_signal_margin_ma": <numeric_value>,41 "fail_closed_cv_margin": <numeric_value>,42 "bypass_cv_margin": <numeric_value>,43 "safe_flow_margin_m3_h": <numeric_value>,44 "safe_runtime_h": <numeric_value>,45 "safe_runtime_margin_h": <numeric_value>,46 "source_resolution_fraction": <numeric_value>,47 "unresolved_conflict_count": <numeric_value>,48 "overall_pass_score": <numeric_value>49}50```51 52Do not claim authority approval, accepted project evidence, safety-case acceptance, executable real source-pack parsing, full standards compliance, generated benchmark readiness, or benchmark readiness.53

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc18 valve failure safe state case

ssc18_valve_failure_safe_state_case

Valve failure safe-state repair source pack

valve-failure-safe-statecontrol-repair-response
Parameter ranges
failed_signal_ma
3.6
fail_threshold_ma
3.8
fail_closed_cv
5
required_safe_cv
4
bypass_cv
12
required_bypass_cv
10
safe_flow_m3_h
15
minimum_safe_flow_m3_h
12
tank_volume_m3
100
drawdown_flow_m3_h
15
required_safe_duration_h
6
source_items_resolved
6
source_items_total
6
unresolved_conflict_count
0

Example task

valve-failure-safe-state-ssc18-valve-failure-safe-state-case-preview — hard difficulty, all inputs given.

Valve failure safe-state repair source pack. valve-failure-safe-state. Required outputs: failed_signal_margin_ma, fail_closed_cv_margin, bypass_cv_margin, safe_flow_margin_m3_h, safe_runtime_h, safe_runtime_margin_h

The model sees

Scenario context and visible inputs.

failed_signal_ma
3.6 to 3.6 mA
fail_threshold_ma
3.8 to 3.8 mA
fail_closed_cv
5 to 5
required_safe_cv
4 to 4
bypass_cv
12 to 12
required_bypass_cv
10 to 10
safe_flow_m3_h
15 to 15 m3/h
minimum_safe_flow_m3_h
12 to 12 m3/h
tank_volume_m3
100 to 100 m3
drawdown_flow_m3_h
15 to 15 m3/h
required_safe_duration_h
6 to 6 h
source_items_resolved
6 to 6
source_items_total
6 to 6
unresolved_conflict_count
0 to 0

Executable tool: valve-failure-safe-state-repair-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "failed_signal_margin_ma": <number>,
  "fail_closed_cv_margin": <number>,
  "bypass_cv_margin": <number>,
  "safe_flow_margin_m3_h": <number>,
  "safe_runtime_h": <number>,
  "safe_runtime_margin_h": <number>,
  "source_resolution_fraction": <number>,
  "unresolved_conflict_count": <number>,
  "overall_pass_score": <number>
}
  • failed_signal_margin_ma · scored within ±3%
  • fail_closed_cv_margin · scored within ±3%
  • bypass_cv_margin · scored within ±3%
  • safe_flow_margin_m3_h · scored within ±3%
  • safe_runtime_h · scored within ±3%
  • safe_runtime_margin_h · scored within ±3%
  • source_resolution_fraction · scored within ±3%
  • unresolved_conflict_count · scored within ±3%
  • overall_pass_score · scored within ±3%