LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
civilwith-tool

Downpipe Sizing

Sizes roof downpipes by calculating the design stormwater flow per downpipe from the catchment area and rainfall intensity, then selecting the smallest standard uPVC round diameter whose full-bore capacity meets or exceeds the demand per AS/NZS 3500.3 Table 4.3. Used in building plumbing and roof drainage design to ensure adequate stormwater disposal from roofs.

with-tool: The model is given an executable Python calculator script.

Standards

AS/NZS 3500.3:2025

Tags

civildrainageplumbingroof-drainagestormwater

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

3 inputs

Always given

Included directly in every task prompt.

2

  • Roof catchment area

    roof_catchment_area_m2

    Effective roof catchment area served by each group of downpipes

    20 – 5000 m²

  • Num downpipes

    num_downpipes

    Number of downpipes draining the catchment area

    1 – 20

Derived from scenario

Sampled from the scenario and inferable from its description.

1

  • Rainfall intensity

    rainfall_intensity_mm_hr

    Design rainfall intensity for the ARI and duration

    Derived from the archetype scenario. Hidden at hard difficulty.

    50 – 300 mm/hr

Scored outputs

4 outputs

Design flow

design_flow_l_s

Design flow per downpipe (L/s)

Scores if within ±3% of the reference value.

Selected diameter

selected_diameter_mm

Selected standard downpipe diameter (mm)

Scores if within ±1% of the reference value.

Selected capacity

selected_capacity_l_s

Full-bore capacity of the selected downpipe (L/s)

Scores if within ±3% of the reference value.

Compliance

compliance

Compliance flag: 1.0 if capacity >= design flow, else 0.0

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Small residential roof, all parameters given — straightforward table lookup

medium

All inputs given

Larger commercial or industrial roof, all parameters given, higher flows

hard

Some inputs hidden

Rainfall intensity hidden — agent must infer I from site context and ARI

Hidden inputs

  • Rainfall intensityrainfall_intensity_mm_hr

Prompt replacement text

The building is {{ archetype.description }} located at {{ archetype.site_context }}

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • downpipe-sizing_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior civil engineer specializing in roof drainage and stormwater plumbing design.2 3## Problem4 5Size downpipes for a roof catchment using the AS/NZS 3500.3 standard capacity table for round uPVC downpipes.6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Roof catchment area (A) | {{ roof_catchment_area_m2 }} | m² |12{% if rainfall_intensity_mm_hr is defined %}13| Design rainfall intensity (I) | {{ rainfall_intensity_mm_hr }} | mm/hr |14{% endif %}15| Number of downpipes | {{ num_downpipes }} | - |16{% if archetype_description is defined %}17 18### Site Conditions19 20{{ archetype_description }}21{% endif %}22 23{% if tool_available %}24## Available Tool25 26A downpipe sizing calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:27 28```bash29python3 /workspace/{{ meta.name }}_calc.py --help30```31 32You may use this tool to verify your calculations or compute values directly.33{% endif %}34 35## Required36 37Calculate the following:38 391. Design flow per downpipe Q (L/s)402. Selected standard downpipe diameter (mm)413. Full-bore capacity of the selected downpipe (L/s)424. Compliance assessment (1.0 = pass, 0.0 = fail)43 44## Applicable Standards45 46- AS/NZS 3500.3:2025 Plumbing and Drainage — Stormwater Drainage47 48## Constraints49 50- No internet access is available. Work from engineering knowledge and the provided tool.51- Use the following design flow formula (SI units):52 - **Design flow per downpipe:** Q = (I × A / N) / 360053 - where I is rainfall intensity (mm/hr), A is catchment area (m²), N is number of downpipes54 - Q is in litres per second (L/s)55- Select the smallest standard uPVC round downpipe diameter from AS/NZS 3500.3 Table 4.3 whose full-bore capacity meets or exceeds the design flow:56 57| Diameter (mm) | Capacity (L/s) |58|---------------|----------------|59| 65 | 0.7 |60| 80 | 1.3 |61| 90 | 2.0 |62| 100 | 3.0 |63| 125 | 5.5 |64| 150 | 9.0 |65 66- Compliance: pass (1.0) if selected capacity >= design flow, fail (0.0) otherwise67 68## Output Format69 70Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:71 72```json73{74 "design_flow_l_s": <numeric_value>,75 "selected_diameter_mm": <numeric_value>,76 "selected_capacity_l_s": <numeric_value>,77 "compliance": <numeric_value>78}79```80 81Write your complete solution to `/workspace/output.md`.82

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Residential house

residential_house

Single-storey or two-storey residential dwelling with pitched roof

sydney-inner-west-housemelbourne-bayside-cottagebrisbane-queenslander
Parameter ranges
roof_catchment_area_m2
50 – 250
num_downpipes
2 – 6

Commercial warehouse

commercial_warehouse

Large commercial warehouse or retail shed with flat or low-pitch metal roof

sydney-industrial-parkmelbourne-western-logisticsbrisbane-trade-coast
Parameter ranges
roof_catchment_area_m2
500 – 5000
num_downpipes
4 – 20

Covered car park

covered_car_park

Multi-level or single-level covered car park with exposed upper deck

perth-cbd-car-parkadelaide-airport-parkinggold-coast-shopping-centre
Parameter ranges
roof_catchment_area_m2
200 – 2000
num_downpipes
4 – 12

Industrial shed

industrial_shed

Industrial manufacturing or storage shed with large-span metal roof

darwin-industrial-estatetownsville-port-shedcairns-processing-facility
Parameter ranges
roof_catchment_area_m2
800 – 4000
num_downpipes
6 – 16

Example task

sydney-inner-west-house-residential-house-preview — hard difficulty, some inputs hidden.

Single-storey or two-storey residential dwelling with pitched roof. sydney-inner-west-house. Required outputs: design_flow_l_s, selected_diameter_mm, selected_capacity_l_s, compliance

The model sees

Scenario context and visible inputs.

roof_catchment_area_m2
50 to 250 m²
num_downpipes
2 to 6 -

Executable tool: downpipe-sizing_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Rainfall intensity

    rainfall_intensity_mm_hr

Stand-in text in the prompt

The building is {{ archetype.description }} located at {{ archetype.site_context }}

The model must produce

The scored JSON answer schema.

{
  "design_flow_l_s": <number>,
  "selected_diameter_mm": <number>,
  "selected_capacity_l_s": <number>,
  "compliance": <number>
}
  • design_flow_l_s · scored within ±3%
  • selected_diameter_mm · scored within ±1%
  • selected_capacity_l_s · scored within ±3%
  • compliance · scored within ±1%