LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Commissioning Calibration Review Package

Calculates source-bound commissioning and calibration metrics from a deterministic SSC-18 task-owned source pack. The template combines ideal 4-20 mA signal, as-found calibration error, loop-test pass fraction, process acceptance margin, valve Cv headroom, and an overall synthetic pass score.

with-tool: The model is given an executable Python calculator script.

Standards

ISA-5calibration sheet workflowloop commissioning workflow

Tags

calibrationcommissioningdeterministicelectricalinstrumentationloop-testvalve

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

11 inputs

Always given

Included directly in every task prompt.

11
Show 11 inputs

  • Test process value

    test_process_value_m3_h

    CAL-18-SHEET-04 test process value

    68.8 m3/h

  • Lower range value

    lower_range_value_m3_h

    LOOP-18-SCHED-04 lower range

    0 m3/h

  • Upper range value

    upper_range_value_m3_h

    LOOP-18-SCHED-04 upper range

    120 m3/h

  • As found signal

    as_found_signal_ma

    CAL-18-SHEET-04 as-found signal

    13.1 mA

  • Calibration tolerance

    calibration_tolerance_ma

    CRIT-18-ACCEPT-04 calibration tolerance

    0.2 mA

  • Loop points passed

    loop_points_passed

    CHECK-18-COMM-04 loop points passed

    12

  • Loop points total

    loop_points_total

    CHECK-18-COMM-04 loop points total

    12

  • Failed point count

    failed_point_count

    CHECK-18-COMM-04 failed point count

    0

  • Process acceptance margin

    process_acceptance_margin

    CRIT-18-ACCEPT-04 process acceptance margin

    1.8

  • Required valve cv

    required_valve_cv

    VDS-18-FCV-04 required valve Cv

    56.464

  • Selected valve cv

    selected_valve_cv

    VDS-18-FCV-04 selected valve Cv

    63

Scored outputs

10 outputs

Ideal signal ma

ideal_signal_ma

Ideal signal at the test process value

Scores if within ±3% of the reference value.

Calibration error ma

calibration_error_ma

As-found signal minus ideal signal

Scores if within ±3% of the reference value.

Calibration error pct span

calibration_error_pct_span

Absolute calibration error as percent of 16 mA span

Scores if within ±3% of the reference value.

Calibration margin ma

calibration_margin_ma

Calibration tolerance minus absolute error

Scores if within ±3% of the reference value.

Loop check pass fraction

loop_check_pass_fraction

Loop points passed divided by total loop points

Scores if within ±3% of the reference value.

Failed point count

failed_point_count

Failed loop/calibration point count

Scores if within ±3% of the reference value.

Process acceptance margin

process_acceptance_margin

Remaining process acceptance margin

Scores if within ±3% of the reference value.

Valve cv headroom

valve_cv_headroom

Selected valve Cv minus required Cv

Scores if within ±3% of the reference value.

Commissioning completeness fraction

commissioning_completeness_fraction

Completeness score for closed commissioning checks

Scores if within ±3% of the reference value.

Overall pass score

overall_pass_score

Pass score for commissioning calibration checks

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-18 commissioning calibration package
medium:
All source-pack values given for the SSC-18 commissioning calibration package
hard:
All source-pack values given for the SSC-18 commissioning calibration package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • commissioning-calibration-review-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are checking `SSC-18-LH-04`, a task-owned synthetic commissioning and calibration review packet.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. Commissioning, calibration, loop test, valve datasheet, and process-acceptance workflows shape the context only; this instance does not parse real commissioning records, calibration sheets, vendor exports, or authority-approved acceptance records.4 5Source pack:6 7- Commissioning checklist: `CHECK-18-COMM-04`8- Calibration sheet: `CAL-18-SHEET-04`9- Valve datasheet: `VDS-18-FCV-04`10- Loop schedule: `LOOP-18-SCHED-04`11- Process acceptance criteria: `CRIT-18-ACCEPT-04`12- Commissioning response: `RESPONSE-18-COMM-04`13 14Given values:15 16| Field | Value |17| --- | ---: |18| Test process value | {{ test_process_value_m3_h }} m3/h |19| Range | {{ lower_range_value_m3_h }} to {{ upper_range_value_m3_h }} m3/h |20| As-found signal | {{ as_found_signal_ma }} mA |21| Calibration tolerance | {{ calibration_tolerance_ma }} mA |22| Loop points passed/total | {{ loop_points_passed }} / {{ loop_points_total }} |23| Failed points | {{ failed_point_count }} |24| Process acceptance margin | {{ process_acceptance_margin }} |25| Required/selected valve Cv | {{ required_valve_cv }} / {{ selected_valve_cv }} |26 27Required calculations:28 29- Ideal signal equals `4 + 16 x (test value - lower range) / span`.30- Calibration error equals as-found signal minus ideal signal.31- Calibration error percent span equals absolute calibration error divided by 16 mA times 100.32- Calibration margin equals tolerance minus absolute calibration error.33- Loop-check pass fraction equals passed points divided by total points.34- Valve Cv headroom equals selected Cv minus required Cv.35 36Return one JSON object with keys:37 38```json39{40 "ideal_signal_ma": <numeric_value>,41 "calibration_error_ma": <numeric_value>,42 "calibration_error_pct_span": <numeric_value>,43 "calibration_margin_ma": <numeric_value>,44 "loop_check_pass_fraction": <numeric_value>,45 "failed_point_count": <numeric_value>,46 "process_acceptance_margin": <numeric_value>,47 "valve_cv_headroom": <numeric_value>,48 "commissioning_completeness_fraction": <numeric_value>,49 "overall_pass_score": <numeric_value>50}51```52 53Do not claim authority approval, accepted project evidence, commissioning acceptance, executable real source-pack parsing, full standards compliance, generated benchmark readiness, or benchmark readiness.54

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc18 commissioning calibration case

ssc18_commissioning_calibration_case

Commissioning calibration and loop-review source pack

commissioning-calibrationloop-test-review
Parameter ranges
test_process_value_m3_h
68.8
lower_range_value_m3_h
0
upper_range_value_m3_h
120
as_found_signal_ma
13.1
calibration_tolerance_ma
0.2
loop_points_passed
12
loop_points_total
12
failed_point_count
0
process_acceptance_margin
1.8
required_valve_cv
56.464
selected_valve_cv
63

Example task

commissioning-calibration-ssc18-commissioning-calibration-case-preview — hard difficulty, all inputs given.

Commissioning calibration and loop-review source pack. commissioning-calibration. Required outputs: ideal_signal_ma, calibration_error_ma, calibration_error_pct_span, calibration_margin_ma, loop_check_pass_fraction, failed_point_count

The model sees

Scenario context and visible inputs.

test_process_value_m3_h
68.8 to 68.8 m3/h
lower_range_value_m3_h
0 to 0 m3/h
upper_range_value_m3_h
120 to 120 m3/h
as_found_signal_ma
13.1 to 13.1 mA
calibration_tolerance_ma
0.2 to 0.2 mA
loop_points_passed
12 to 12
loop_points_total
12 to 12
failed_point_count
0 to 0
process_acceptance_margin
1.8 to 1.8
required_valve_cv
56.464 to 56.464
selected_valve_cv
63 to 63

Executable tool: commissioning-calibration-review-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "ideal_signal_ma": <number>,
  "calibration_error_ma": <number>,
  "calibration_error_pct_span": <number>,
  "calibration_margin_ma": <number>,
  "loop_check_pass_fraction": <number>,
  "failed_point_count": <number>,
  "process_acceptance_margin": <number>,
  "valve_cv_headroom": <number>,
  "commissioning_completeness_fraction": <number>,
  "overall_pass_score": <number>
}
  • ideal_signal_ma · scored within ±3%
  • calibration_error_ma · scored within ±3%
  • calibration_error_pct_span · scored within ±3%
  • calibration_margin_ma · scored within ±3%
  • loop_check_pass_fraction · scored within ±3%
  • failed_point_count · scored within ±3%
  • process_acceptance_margin · scored within ±3%
  • valve_cv_headroom · scored within ±3%
  • commissioning_completeness_fraction · scored within ±3%
  • overall_pass_score · scored within ±3%