LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Warning Time Calculation

Calculates total level crossing warning time by summing minimum road-user warning time, clearance time, barrier lowering time, and system delay. The template converts maximum train speed to m/s and computes the required strike-in detection distance.

with-tool: The model is given an executable Python calculator script.

Standards

AS 7711Network Rail standards

Tags

deterministicelectricallevel-crossingrailsignallingwarning-time

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

5 inputs

Always given

Included directly in every task prompt.

4

  • Maximum train speed kmh

    maximum_train_speed_kmh

    Maximum train speed

    5 – 200 km/h

  • Minimum warning time

    minimum_warning_time_s

    Minimum road-user warning time

    5 – 120 s

  • Road user clearance time

    road_user_clearance_time_s

    Road user clearance time

    0 – 60 s

  • Barrier lowering time

    barrier_lowering_time_s

    Barrier lowering or equipment operating time

    0 – 30 s

Hidden at higher difficulty

Visible in easier tasks and withheld in one or more harder tiers.

1

  • System delay

    system_delay_s

    Signal processing and system delay

    Hidden at hard difficulty.

    0 – 20 s

Scored outputs

4 outputs

Maximum train speed m s

maximum_train_speed_m_s

Maximum train speed converted to m/s

Scores if within ±1% of the reference value.

Total warning time s

total_warning_time_s

Total warning time used for strike-in

Scores if within ±3% of the reference value.

Strike in distance

strike_in_distance_m

Strike-in detection distance

Scores if within ±3% of the reference value.

Minimum warning margin s

minimum_warning_margin_s

Additional time above the minimum warning time

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Low-speed crossing with all timing values visible

medium

All inputs given

Low-speed or mainline crossing

hard

Some inputs hidden

System delay hidden in signalling context

Hidden inputs

  • System delay ssystem_delay_s

Prompt replacement text

Use the signalling system delay implied by {{ archetype.description }}.

System delay s restricted to: 3

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • warning-time-calculation_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior rail signalling engineer calculating level crossing strike-in distance.2 3## Given4 5| Parameter | Value | Unit |6|-----------|-------|------|7| Maximum train speed | {{ maximum_train_speed_kmh }} | km/h |8| Minimum warning time | {{ minimum_warning_time_s }} | s |9| Road user clearance time | {{ road_user_clearance_time_s }} | s |10| Barrier lowering time | {{ barrier_lowering_time_s }} | s |11{% if system_delay_s is defined %}12| System delay | {{ system_delay_s }} | s |13{% endif %}14 15## Constraints16 17- Convert train speed from km/h to m/s.18- Total warning time equals the sum of all timing components.19- Strike-in distance equals train speed in m/s times total warning time.20- Minimum warning margin equals total warning time minus minimum warning time.21 22## Output Format23 24Include a JSON block with exactly these keys:25 26```json27{28 "maximum_train_speed_m_s": <numeric_value>,29 "total_warning_time_s": <numeric_value>,30 "strike_in_distance_m": <numeric_value>,31 "minimum_warning_margin_s": <numeric_value>32}33```34 35Write your complete solution to `/workspace/output.md`.36

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Low speed crossing

low_speed_crossing

Low-speed urban or yard crossing

urban-crossingyard-access
Parameter ranges
maximum_train_speed_kmh
20 – 60
minimum_warning_time_s
20 – 35
road_user_clearance_time_s
3 – 10
barrier_lowering_time_s
0 – 8
system_delay_s
1 – 5

Mainline crossing

mainline_crossing

Mainline level crossing with active controls

regional-mainlinesuburban-mainline
Parameter ranges
maximum_train_speed_kmh
60 – 160
minimum_warning_time_s
20 – 45
road_user_clearance_time_s
5 – 15
barrier_lowering_time_s
5 – 12
system_delay_s
1 – 8

Example task

regional-mainline-mainline-crossing-preview — hard difficulty, some inputs hidden.

Mainline level crossing with active controls. regional-mainline. Required outputs: maximum_train_speed_m_s, total_warning_time_s, strike_in_distance_m, minimum_warning_margin_s

The model sees

Scenario context and visible inputs.

maximum_train_speed_kmh
60 to 160 km/h
minimum_warning_time_s
20 to 45 s
road_user_clearance_time_s
5 to 15 s
barrier_lowering_time_s
5 to 12 s

Executable tool: warning-time-calculation_calc.py

The model must infer

Inputs withheld at this difficulty.

  • System delay s

    system_delay_s

Stand-in text in the prompt

Use the signalling system delay implied by {{ archetype.description }}.

The model must produce

The scored JSON answer schema.

{
  "maximum_train_speed_m_s": <number>,
  "total_warning_time_s": <number>,
  "strike_in_distance_m": <number>,
  "minimum_warning_margin_s": <number>
}
  • maximum_train_speed_m_s · scored within ±1%
  • total_warning_time_s · scored within ±3%
  • strike_in_distance_m · scored within ±3%
  • minimum_warning_margin_s · scored within ±3%