LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Road Uniformity Check

Calculates overall and longitudinal road lighting uniformity ratios and the margin against a target overall uniformity class requirement.

with-tool: The model is given an executable Python calculator script.

Standards

AS/NZS 1158.2EN 13201-2

Tags

deterministicelectricallightingroad-lightinguniformity

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

5 inputs

Always given

Included directly in every task prompt.

4

  • Minimum luminance

    minimum_luminance_cd_m2

    Minimum calculated luminance on the road grid

    0 – 5 cd/m2

  • Average luminance

    average_luminance_cd_m2

    Average calculated luminance on the road grid

    0.1 – 10 cd/m2

  • Longitudinal min luminance

    longitudinal_min_luminance_cd_m2

    Minimum luminance along the relevant lane line

    0 – 5 cd/m2

  • Longitudinal max luminance

    longitudinal_max_luminance_cd_m2

    Maximum luminance along the relevant lane line

    0.1 – 10 cd/m2

Hidden at higher difficulty

Visible in easier tasks and withheld in one or more harder tiers.

1

  • Target overall uniformity

    target_overall_uniformity

    Target overall uniformity ratio

    Hidden at hard difficulty.

    0.1 – 0.8

Scored outputs

3 outputs

Overall uniformity uo

overall_uniformity_uo

Overall uniformity ratio

Scores if within ±3% of the reference value.

Longitudinal uniformity ul

longitudinal_uniformity_ul

Longitudinal uniformity ratio

Scores if within ±3% of the reference value.

Overall uniformity margin pct

overall_uniformity_margin_pct

Percentage margin against target overall uniformity

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Local road grid with all values visible

medium

All inputs given

Road lighting grid selected from local or arterial cases

hard

Some inputs hidden

Arterial road grid with target class embedded in context

Hidden inputs

  • Target overall uniformitytarget_overall_uniformity

Prompt replacement text

Use the target overall uniformity ratio implied by {{ archetype.description }}.

Target overall uniformity restricted to: 0.4

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • road-uniformity-check_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1# ABOUTME: Prompt template for road lighting uniformity check tasks.2# ABOUTME: Presents luminance values and target uniformity for calculation.3 4You are a senior road lighting engineer verifying lighting uniformity.5 6## Given7 8| Parameter | Value | Unit |9|-----------|-------|------|10| Minimum luminance | {{ minimum_luminance_cd_m2 }} | cd/m2 |11| Average luminance | {{ average_luminance_cd_m2 }} | cd/m2 |12| Longitudinal minimum luminance | {{ longitudinal_min_luminance_cd_m2 }} | cd/m2 |13| Longitudinal maximum luminance | {{ longitudinal_max_luminance_cd_m2 }} | cd/m2 |14| Target overall uniformity | {{ target_overall_uniformity }} | - |15 16## Constraints17 18- Overall uniformity Uo equals minimum luminance divided by average luminance.19- Longitudinal uniformity Ul equals longitudinal minimum luminance divided by longitudinal maximum luminance.20- Overall uniformity margin is the percentage difference from the target overall uniformity.21 22## Output Format23 24Include a JSON block with exactly these keys:25 26```json27{28 "overall_uniformity_uo": <numeric_value>,29 "longitudinal_uniformity_ul": <numeric_value>,30 "overall_uniformity_margin_pct": <numeric_value>31}32```33 34Write your complete solution to `/workspace/output.md`.35

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Local road grid

local_road_grid

Local road lighting calculation grid

local-roadlighting-grid
Parameter ranges
minimum_luminance_cd_m2
0.2 – 0.8
average_luminance_cd_m2
0.8 – 2
longitudinal_min_luminance_cd_m2
0.3 – 1
longitudinal_max_luminance_cd_m2
0.8 – 2.5
target_overall_uniformity
0.3 – 0.5

Arterial road grid

arterial_road_grid

Arterial road luminance calculation grid

arterial-roadlighting-grid
Parameter ranges
minimum_luminance_cd_m2
0.5 – 1.5
average_luminance_cd_m2
1.2 – 4
longitudinal_min_luminance_cd_m2
0.6 – 2
longitudinal_max_luminance_cd_m2
1.2 – 5
target_overall_uniformity
0.35 – 0.5

Example task

arterial-road-arterial-road-grid-preview — hard difficulty, some inputs hidden.

Arterial road luminance calculation grid. arterial-road. Required outputs: overall_uniformity_uo, longitudinal_uniformity_ul, overall_uniformity_margin_pct

The model sees

Scenario context and visible inputs.

minimum_luminance_cd_m2
0.5 to 1.5 cd/m2
average_luminance_cd_m2
1.2 to 4 cd/m2
longitudinal_min_luminance_cd_m2
0.6 to 2 cd/m2
longitudinal_max_luminance_cd_m2
1.2 to 5 cd/m2

Executable tool: road-uniformity-check_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Target overall uniformity

    target_overall_uniformity

Stand-in text in the prompt

Use the target overall uniformity ratio implied by {{ archetype.description }}.

The model must produce

The scored JSON answer schema.

{
  "overall_uniformity_uo": <number>,
  "longitudinal_uniformity_ul": <number>,
  "overall_uniformity_margin_pct": <number>
}
  • overall_uniformity_uo · scored within ±3%
  • longitudinal_uniformity_ul · scored within ±3%
  • overall_uniformity_margin_pct · scored within ±3%