LIVEdataset aec-bench@releasetasks 552models 18disciplines 5last submission … · built …

civilwith-tool

Multimodal Corridor Review Response Package

Calculates task-owned synthetic SSC-01 review-response metrics for changed chainage, HGL clearance, pedestrian clearance, VMS legibility, feeder voltage drop, and comment closeout.

with-tool: The model is given an executable Python calculator script.

Standards

FHWA MUTCD corridor review workflow drainage design workflow

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

01
Template
The reusable contract shown on this page.
02
Scenario
An archetype and site context are sampled.
03
Difficulty tier
Inputs may be hidden at harder tiers.
04
Task prompt
The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

22 inputs

Always given

Included directly in every task prompt.

Show 22 inputs

Original chainage
original_chainage_m
MARKUP-SSC01-008 original comment chainage
1280 m
Revised chainage
revised_chainage_m
MARKUP-SSC01-008 revised comment chainage
1292.5 m
Revised road level
revised_road_level_m
DRAIN-SSC01-008 revised road level
42.35 m
Revised hgl
revised_hgl_m
DRAIN-SSC01-008 revised HGL
42.02 m
Minimum hgl clearance
minimum_hgl_clearance_mm
DRAIN-SSC01-008 minimum HGL clearance
150 mm
Pedestrian startup time
pedestrian_startup_time_s
SIG-SSC01-008 pedestrian startup allowance
3 s
Revised crossing width
revised_crossing_width_m
SIG-SSC01-008 revised crossing width
16 m
Pedestrian walk speed
pedestrian_walk_speed_m_s
SIG-SSC01-008 pedestrian walk speed
1.2 m/s
Available ped clearance
available_ped_clearance_s
SIG-SSC01-008 available pedestrian clearance
20 s
Vms character height
vms_character_height_in
FEED-SSC01-008 revised VMS character height
10 in
Approach speed kmh
approach_speed_kmh
SIG-SSC01-008 revised approach speed
50 km/h
Reading rate
reading_rate_chars_s
SIG-SSC01-008 reading rate
4 chars/s
Revised message length
revised_message_length_chars
FEED-SSC01-008 revised VMS message length
25 chars
Revised device load
revised_device_load_w
FEED-SSC01-008 revised connected device load
760 W
Feeder length
feeder_length_km
FEED-SSC01-008 revised feeder length
0.22 km
Conductor resistance
conductor_resistance_ohm_km
FEED-SSC01-008 conductor resistance
0.5 ohm/km
Feeder voltage
feeder_voltage_v
FEED-SSC01-008 feeder voltage
230 V
Power factor
power_factor
FEED-SSC01-008 load power factor
0.9
Allowable voltage drop pct
allowable_voltage_drop_pct
FEED-SSC01-008 allowable voltage drop
5 %
Review comments total
review_comments_total
REV-SSC01-008 total comments in the response batch
6
Review comments closed
review_comments_closed
REV-SSC01-008 closed comments in the response batch
6
Impacted calculation count
impacted_calculation_count
REV-SSC01-008 count of recalculations touched by the change
5

Scored outputs

12 outputs

Changed chainage delta

changed_chainage_delta_m

Revised chainage minus original chainage

Scores if within ±3% of the reference value.

Hgl clearance

hgl_clearance_mm

Revised road level minus revised HGL

Scores if within ±50% of the reference value.

Hgl clearance margin

hgl_clearance_margin_mm

HGL clearance minus minimum required clearance

Scores if within ±50% of the reference value.

Ped clearance required s

ped_clearance_required_s

Required pedestrian clearance time

Scores if within ±3% of the reference value.

Ped clearance margin s

ped_clearance_margin_s

Available pedestrian clearance minus required clearance

Scores if within ±3% of the reference value.

Vms reading time s

vms_reading_time_s

Available VMS reading time

Scores if within ±3% of the reference value.

Vms message margin chars

vms_message_margin_chars

Readable character capacity minus revised message length

Scores if within ±3% of the reference value.

Feeder voltage drop

feeder_voltage_drop_percent

Voltage drop at revised device load

Scores if within ±3% of the reference value.

Voltage drop margin

voltage_drop_margin_percent

Allowable voltage drop minus computed voltage drop

Scores if within ±3% of the reference value.

Comment closeout

comment_closeout_percent

Closed comments divided by total comments

Scores if within ±3% of the reference value.

Impacted calculation count

impacted_calculation_count

Number of discipline calculations impacted by the review change

Scores if within ±1% of the reference value.

Overall pass score

overall_pass_score

1.0 when review-response drainage, timing, VMS, voltage, and comment checks pass

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:: All source-pack values visible
medium:: Same deterministic source-pack values with package-level aggregation
hard:: Same deterministic source-pack values with cross-discipline handoff checks

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

instruction.md
multimodal-corridor-review-response-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a corridor package reviewer checking a task-owned synthetic SSC-01 multimodal corridor review response.2 3Use only the task-owned synthetic source pack values below for numeric grading. Road, drainage, traffic, ITS, and electrical review workflows shape the context only; they are not extra data sources for this instance.4 5## Scene6 7- Product: `SSC-01-LH-08`8- Review comment register: `REV-SSC01-008`9- Marked-up corridor plan: `MARKUP-SSC01-008`10- Revised drainage calculation: `DRAIN-SSC01-008`11- Revised signal and pedestrian timing sheet: `SIG-SSC01-008`12- Revised feeder schedule: `FEED-SSC01-008`13- Response memo: `MEMO-SSC01-008`14 15## Source Values16 17- Original and revised chainage: {{ original_chainage_m }} m and {{ revised_chainage_m }} m18- Revised road level and HGL: {{ revised_road_level_m }} m and {{ revised_hgl_m }} m19- Minimum HGL clearance: {{ minimum_hgl_clearance_mm }} mm20- Pedestrian startup, crossing width, walk speed, and available clearance: {{ pedestrian_startup_time_s }} s, {{ revised_crossing_width_m }} m, {{ pedestrian_walk_speed_m_s }} m/s, {{ available_ped_clearance_s }} s21- VMS character height, approach speed, reading rate, and message length: {{ vms_character_height_in }} in, {{ approach_speed_kmh }} km/h, {{ reading_rate_chars_s }} chars/s, {{ revised_message_length_chars }} chars22- Revised device load: {{ revised_device_load_w }} W23- Feeder length, resistance, voltage, power factor, and voltage-drop limit: {{ feeder_length_km }} km, {{ conductor_resistance_ohm_km }} ohm/km, {{ feeder_voltage_v }} V, {{ power_factor }}, {{ allowable_voltage_drop_pct }} %24- Review comments closed/total: {{ review_comments_closed }} / {{ review_comments_total }}25- Impacted calculations: {{ impacted_calculation_count }}26 27## Required Calculations28 29- Changed chainage delta is revised chainage minus original chainage.30- HGL clearance is revised road level minus revised HGL, converted to mm.31- Pedestrian clearance required is startup time plus crossing width divided by walk speed.32- VMS reading time is character height times 40 ft/in, converted to metres, divided by approach speed.33- Feeder voltage drop is `2 x length x resistance x current / voltage x 100`.34- Comment closeout percent is closed comments divided by total comments.35- Overall pass score is `1.0` only when drainage, pedestrian, VMS, voltage, and review-closeout checks pass.36 37Write a compact memo to `/workspace/output.md`. Include a source-boundary statement that this is a task-owned synthetic source pack. Preserve the source IDs above and state that the baseline source pack passes the current synthetic checks.38 39Do not claim authority approval, accepted project evidence, software export validity, full standards compliance, executable source-pack hardening, generated benchmark readiness, or benchmark readiness.40 41Include a fenced JSON block with exactly these numeric keys:42 43```json44{45  "changed_chainage_delta_m": <numeric_value>,46  "hgl_clearance_mm": <numeric_value>,47  "hgl_clearance_margin_mm": <numeric_value>,48  "ped_clearance_required_s": <numeric_value>,49  "ped_clearance_margin_s": <numeric_value>,50  "vms_reading_time_s": <numeric_value>,51  "vms_message_margin_chars": <numeric_value>,52  "feeder_voltage_drop_percent": <numeric_value>,53  "voltage_drop_margin_percent": <numeric_value>,54  "comment_closeout_percent": <numeric_value>,55  "impacted_calculation_count": <numeric_value>,56  "overall_pass_score": <numeric_value>57}58```59

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc01 multimodal review

ssc01_multimodal_review

SSC-01 task-owned synthetic multimodal corridor review response package

multimodal-corridor-review-response

Parameter ranges

Example task

multimodal-corridor-review-response-ssc01-multimodal-review-preview — hard difficulty, all inputs given.

SSC-01 task-owned synthetic multimodal corridor review response package. multimodal-corridor-review-response. Required outputs: changed_chainage_delta_m, hgl_clearance_mm, hgl_clearance_margin_mm, ped_clearance_required_s, ped_clearance_margin_s, vms_reading_time_s

The model sees

Scenario context and visible inputs.

original_chainage_m: 1280 to 1280 m
revised_chainage_m: 1292.5 to 1292.5 m
revised_road_level_m: 42.35 to 42.35 m
revised_hgl_m: 42.02 to 42.02 m
minimum_hgl_clearance_mm: 150 to 150 mm
pedestrian_startup_time_s: 3 to 3 s
revised_crossing_width_m: 16 to 16 m
pedestrian_walk_speed_m_s: 1.2 to 1.2 m/s
available_ped_clearance_s: 20 to 20 s
vms_character_height_in: 10 to 10 in
approach_speed_kmh: 50 to 50 km/h
reading_rate_chars_s: 4 to 4 chars/s
revised_message_length_chars: 25 to 25 chars
revised_device_load_w: 760 to 760 W
feeder_length_km: 0.22 to 0.22 km
conductor_resistance_ohm_km: 0.5 to 0.5 ohm/km
feeder_voltage_v: 230 to 230 V
power_factor: 0.9 to 0.9
allowable_voltage_drop_pct: 5 to 5 %
review_comments_total: 6 to 6
review_comments_closed: 6 to 6
impacted_calculation_count: 5 to 5

Executable tool: multimodal-corridor-review-response-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "changed_chainage_delta_m": <number>,
  "hgl_clearance_mm": <number>,
  "hgl_clearance_margin_mm": <number>,
  "ped_clearance_required_s": <number>,
  "ped_clearance_margin_s": <number>,
  "vms_reading_time_s": <number>,
  "vms_message_margin_chars": <number>,
  "feeder_voltage_drop_percent": <number>,
  "voltage_drop_margin_percent": <number>,
  "comment_closeout_percent": <number>,
  "impacted_calculation_count": <number>,
  "overall_pass_score": <number>
}

changed_chainage_delta_m · scored within ±3%
hgl_clearance_mm · scored within ±50%
hgl_clearance_margin_mm · scored within ±50%
ped_clearance_required_s · scored within ±3%
ped_clearance_margin_s · scored within ±3%
vms_reading_time_s · scored within ±3%
vms_message_margin_chars · scored within ±3%
feeder_voltage_drop_percent · scored within ±3%
voltage_drop_margin_percent · scored within ±3%
comment_closeout_percent · scored within ±3%
impacted_calculation_count · scored within ±1%
overall_pass_score · scored within ±1%

Multimodal Corridor Review Response Package

How this task is generated

Template

Scenario

Difficulty tier

Task prompt

Parameters

Inputs

Scored outputs

Changed chainage delta

Hgl clearance

Hgl clearance margin

Ped clearance required s

Ped clearance margin s

Vms reading time s

Vms message margin chars

Feeder voltage drop

Voltage drop margin

Comment closeout

Impacted calculation count

Overall pass score

Difficulty

All inputs remain visible at every tier

Task bundle

Scenario archetypes

Ssc01 multimodal review

Example task

The model sees

The model must infer

The model must produce