LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Vms Legibility Distance

Calculates variable message sign legibility distance from character height using a reduced 40 ft per inch rule of thumb. It converts design speed to ft/s, estimates reading time available, and reports message character capacity from an explicit reading rate.

with-tool: The model is given an executable Python calculator script.

Standards

MUTCDNYSDOT VMS Guidelines

Tags

deterministicelectricalitslegibilitysignagevms

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

3 inputs

Always given

Included directly in every task prompt.

2

  • Character height

    character_height_in

    VMS character height

    4 – 30 in

  • Design speed

    design_speed_mph

    Approach design speed

    10 – 85 mph

Hidden at higher difficulty

Visible in easier tasks and withheld in one or more harder tiers.

1

  • Reading rate

    reading_rate_chars_s

    Assumed driver reading rate

    Hidden at hard difficulty.

    1 – 6 chars/s

Scored outputs

4 outputs

Minimum legibility distance ft

minimum_legibility_distance_ft

Minimum legibility distance

Scores if within ±3% of the reference value.

Design speed ft s

design_speed_ft_s

Design speed converted to feet per second

Scores if within ±1% of the reference value.

Reading time available s

reading_time_available_s

Reading time available over the legibility distance

Scores if within ±3% of the reference value.

Message length limit chars

message_length_limit_chars

Message length supported by available reading time

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Urban VMS with all readability inputs visible

medium

All inputs given

Urban or freeway VMS

hard

Some inputs hidden

Reading rate hidden in driver-readability context

Hidden inputs

  • Reading rate chars sreading_rate_chars_s

Prompt replacement text

Use the driver reading rate implied by {{ archetype.description }}.

Reading rate chars s restricted to: 3

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • vms-legibility-distance_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior ITS engineer checking variable message sign readability.2 3## Given4 5| Parameter | Value | Unit |6|-----------|-------|------|7| Character height | {{ character_height_in }} | in |8| Design speed | {{ design_speed_mph }} | mph |9{% if reading_rate_chars_s is defined %}10| Reading rate | {{ reading_rate_chars_s }} | chars/s |11{% endif %}12 13## Constraints14 15- Use `minimum_legibility_distance = character_height_in x 40 ft`.16- Convert design speed from mph to ft/s.17- Reading time available equals legibility distance divided by speed.18- Message length limit equals reading time times reading rate.19 20## Output Format21 22Include a JSON block with exactly these keys:23 24```json25{26 "minimum_legibility_distance_ft": <numeric_value>,27 "design_speed_ft_s": <numeric_value>,28 "reading_time_available_s": <numeric_value>,29 "message_length_limit_chars": <numeric_value>30}31```32 33Write your complete solution to `/workspace/output.md`.34

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Urban vms

urban_vms

Urban variable message sign on a lower-speed road

urban-arterialtunnel-approach
Parameter ranges
character_height_in
6 – 14
design_speed_mph
25 – 45
reading_rate_chars_s
2 – 4

Freeway vms

freeway_vms

Freeway variable message sign on a high-speed road

motorway-mainlinemanaged-freeway
Parameter ranges
character_height_in
12 – 24
design_speed_mph
50 – 75
reading_rate_chars_s
2 – 4

Example task

motorway-mainline-freeway-vms-preview — hard difficulty, some inputs hidden.

Freeway variable message sign on a high-speed road. motorway-mainline. Required outputs: minimum_legibility_distance_ft, design_speed_ft_s, reading_time_available_s, message_length_limit_chars

The model sees

Scenario context and visible inputs.

character_height_in
12 to 24 in
design_speed_mph
50 to 75 mph

Executable tool: vms-legibility-distance_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Reading rate chars s

    reading_rate_chars_s

Stand-in text in the prompt

Use the driver reading rate implied by {{ archetype.description }}.

The model must produce

The scored JSON answer schema.

{
  "minimum_legibility_distance_ft": <number>,
  "design_speed_ft_s": <number>,
  "reading_time_available_s": <number>,
  "message_length_limit_chars": <number>
}
  • minimum_legibility_distance_ft · scored within ±3%
  • design_speed_ft_s · scored within ±1%
  • reading_time_available_s · scored within ±3%
  • message_length_limit_chars · scored within ±3%