LIVEdataset aec-bench@releasetasks 552models 18disciplines 5last submission … · built …

civilwith-tool

Driveway Gradient Check

Calculates the driveway gradient as a percentage from the level difference and horizontal distance, then checks compliance against location-specific maximum gradients defined in AS/NZS 2890.1:2004 and typical Australian council development control plans. Covers transition zones, residential and commercial internal sections, garage approaches, and shared pedestrian/vehicle paths.

with-tool: The model is given an executable Python calculator script.

Standards

AS/NZS 2890.1:2004 Local Council DCPs

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

01
Template
The reusable contract shown on this page.
02
Scenario
An archetype and site context are sampled.
03
Difficulty tier
Inputs may be hidden at harder tiers.
04
Task prompt
The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

4 inputs

Always given

Included directly in every task prompt.

Start level m
start_level_m
Start level of the driveway section (reduced level)
0 – 200 m AHD
End level m
end_level_m
End level of the driveway section (reduced level)
0 – 200 m AHD
Horizontal length
horizontal_length_m
Horizontal distance of the driveway section
1 – 50 m

Derived from scenario

Sampled from the scenario and inferable from its description.

Location type
location_type
Driveway section location type determining the maximum allowable gradient
Derived from the archetype scenario. Hidden at hard difficulty.
transition_zoneinternal_residentialinternal_commercialnear_garagepedestrian_shared

Scored outputs

3 outputs

Gradient pct

gradient_pct

Calculated driveway gradient (%)

Scores if within ±3% of the reference value.

Max allowable gradient pct

max_allowable_gradient_pct

Maximum allowable gradient for the location type (%)

Scores if within ±1% of the reference value.

Compliance

compliance

Compliance with maximum gradient (1.0 = pass, 0.0 = fail)

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Flat suburban site, all parameters given, transition zone only

Location type restricted to: transition_zone

medium

All inputs given

Any archetype, all parameters given, any location type

hard

Some inputs hidden

Location type hidden, agent must infer from site and driveway description

Hidden inputs

Location typelocation_type

Prompt replacement text

The driveway is at a {{ archetype.description }} site located in {{ archetype.site_context }}

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

instruction.md
driveway-gradient-check_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior civil engineer specializing in driveway and access design for residential and commercial developments.2 3## Problem4 5Calculate the gradient of a driveway section and verify compliance with maximum allowable slopes per AS/NZS 2890.1:2004 and typical Australian council Development Control Plans (DCPs).6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Start level | {{ start_level_m }} | m AHD |12| End level | {{ end_level_m }} | m AHD |13| Horizontal length | {{ horizontal_length_m }} | m |14{%- if location_type is defined %}15| Location type | {{ location_type }} | - |16{%- endif %}17{%- if archetype_description is defined %}18 19### Site Conditions20 21{{ archetype_description }}22{% endif %}23 24{% if tool_available %}25## Available Tool26 27A driveway gradient calculation tool is available at `/workspace/driveway-gradient-check_calc.py`. Run it with:28 29```bash30python3 /workspace/driveway-gradient-check_calc.py --help31```32 33You may use this tool to verify your calculations or compute values directly.34{% endif %}35 36## Required37 38Calculate the following:39 401. Driveway gradient (%)412. Maximum allowable gradient for the location type (%)423. Compliance (1.0 if gradient is within the allowable limit, 0.0 if it exceeds it)43 44## Applicable Standards45 46- AS/NZS 2890.1:2004 — Parking facilities: Off-street car parking47- Local Council DCPs — Development Control Plans for driveway gradients48 49## Maximum Allowable Gradients (AS/NZS 2890.1 & Typical Council DCPs)50 51| Location Type | Maximum Gradient (%) | Ratio |52|---------------|---------------------|-------|53| Transition zone (first 6m from street) | 12.5 | 1:8 |54| Internal residential driveway | 25.0 | 1:4 |55| Internal commercial driveway | 20.0 | 1:5 |56| Near garage / parking area | 16.67 | 1:6 |57| Pedestrian shared access | 12.5 | 1:8 |58 59## Constraints60 61- No internet access is available. Work from engineering knowledge and the provided tool.62- Use the following formula:63  - **Gradient (%):** G = |end_level - start_level| / horizontal_length x 10064  - Levels are in metres AHD; horizontal length is in metres65- Compliance: 1.0 if the calculated gradient is less than or equal to the maximum allowable gradient for the given location type, otherwise 0.066 67## Output Format68 69Show your step-by-step working in Markdown, including the level difference, gradient calculation, identification of the maximum allowable gradient, and compliance assessment. At the end of your solution, include a JSON block with your final answers in exactly this format:70 71```json72{73  "gradient_pct": <numeric_value>,74  "max_allowable_gradient_pct": <numeric_value>,75  "compliance": <numeric_value>76}77```78 79Write your complete solution to `/workspace/output.md`.80

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Steep hillside house

steep_hillside_house

Steep hillside residential driveway with significant level change

sydney-north-shore-hillsidehobart-steep-residential

Parameter ranges

start_level_m: 32 – 48
end_level_m: 34 – 52
horizontal_length_m: 8 – 20
location_type: internal_residential, near_garage

Flat suburban

flat_suburban

Flat suburban residential driveway with gentle grade

brisbane-suburban-flatadelaide-plains-residential

Parameter ranges

start_level_m: 14 – 16
end_level_m: 14 – 16.5
horizontal_length_m: 5 – 12
location_type: transition_zone, internal_residential

Commercial carpark

commercial_carpark

Commercial car park driveway access from street to parking level

melbourne-cbd-carparkperth-commercial-precinct

Parameter ranges

start_level_m: 18 – 22
end_level_m: 18.5 – 24
horizontal_length_m: 8 – 30
location_type: transition_zone, internal_commercial, near_garage

Shared access

shared_access

Shared pedestrian and vehicle access path in a mixed-use development

canberra-mixed-use-developmentgold-coast-shared-access

Parameter ranges

start_level_m: 10 – 12
end_level_m: 10 – 12.5
horizontal_length_m: 6 – 15
location_type: pedestrian_shared, transition_zone

Example task

sydney-north-shore-hillside-steep-hillside-house-preview — hard difficulty, some inputs hidden.

Steep hillside residential driveway with significant level change. sydney-north-shore-hillside. Required outputs: gradient_pct, max_allowable_gradient_pct, compliance

The model sees

Scenario context and visible inputs.

start_level_m: 32 to 48 m AHD
end_level_m: 34 to 52 m AHD
horizontal_length_m: 8 to 20 m

Executable tool: driveway-gradient-check_calc.py

The model must infer

Inputs withheld at this difficulty.

Location type
location_type

Stand-in text in the prompt

The driveway is at a {{ archetype.description }} site located in {{ archetype.site_context }}

The model must produce

The scored JSON answer schema.

{
  "gradient_pct": <number>,
  "max_allowable_gradient_pct": <number>,
  "compliance": <number>
}

gradient_pct · scored within ±3%
max_allowable_gradient_pct · scored within ±1%
compliance · scored within ±1%

Driveway Gradient Check

How this task is generated

Template

Scenario

Difficulty tier

Task prompt

Parameters

Inputs

Scored outputs

Gradient pct

Max allowable gradient pct

Compliance

Difficulty

easy

medium

hard

Task bundle

Scenario archetypes

Steep hillside house

Flat suburban

Commercial carpark

Shared access

Example task

The model sees

The model must infer

The model must produce