LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
civilwith-tool

Gutter Sizing

Sizes eaves gutters by computing the design stormwater flow from the roof catchment area and rainfall intensity, then selecting the smallest standard gutter profile whose capacity (scaled from the AS/NZS 3500.3 Table 4.2 reference grade by the square root of the installed grade ratio) meets or exceeds the demand. Used in building roof drainage design to prevent overflow and water damage.

with-tool: The model is given an executable Python calculator script.

Standards

AS/NZS 3500.3:2025

Tags

civildrainagegutterplumbingroof-drainagestormwater

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

4 inputs

Always given

Included directly in every task prompt.

3

  • Roof catchment area

    roof_catchment_area_m2

    Effective roof catchment area draining to the gutter run

    10 – 5000 m²

  • Gutter profile

    gutter_profile

    Nominated eaves gutter profile

    100mm_quad115mm_quad125mm_half_round150mm_quad150mm_half_round175mm_OG

  • Gutter grade pct

    gutter_grade_pct

    Longitudinal grade of the gutter (percentage)

    0.1 – 2 %

Derived from scenario

Sampled from the scenario and inferable from its description.

1

  • Rainfall intensity

    rainfall_intensity_mm_hr

    Design rainfall intensity for the ARI and duration

    Derived from the archetype scenario. Hidden at hard difficulty.

    50 – 300 mm/hr

Scored outputs

4 outputs

Design flow

design_flow_l_s

Design stormwater flow from the catchment (L/s)

Scores if within ±3% of the reference value.

Gutter capacity

gutter_capacity_l_s

Capacity of the selected gutter at the installed grade (L/s)

Scores if within ±5% of the reference value.

Required gutter size

required_gutter_size_mm

Nominal size of the smallest adequate standard gutter (mm)

Scores if within ±1% of the reference value.

Compliance

compliance

Compliance flag: 1.0 if capacity >= design flow, else 0.0

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Small residential roof, all parameters given — straightforward capacity check

medium

All inputs given

Larger commercial or industrial roof, all parameters given, higher flows may require upsizing

hard

Some inputs hidden

Rainfall intensity hidden — agent must infer I from site context and ARI

Hidden inputs

  • Rainfall intensityrainfall_intensity_mm_hr

Prompt replacement text

The building is {{ archetype.description }} located at {{ archetype.site_context }}

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • gutter-sizing_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior civil engineer specializing in roof drainage and stormwater plumbing design.2 3## Problem4 5Size an eaves gutter for a roof catchment using the AS/NZS 3500.3 standard gutter capacity table.6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Roof catchment area (A) | {{ roof_catchment_area_m2 }} | m² |12{% if rainfall_intensity_mm_hr is defined %}13| Design rainfall intensity (I) | {{ rainfall_intensity_mm_hr }} | mm/hr |14{% endif %}15| Nominated gutter profile | {{ gutter_profile }} | - |16| Gutter grade | {{ gutter_grade_pct }} | % |17{% if archetype_description is defined %}18 19### Site Conditions20 21{{ archetype_description }}22{% endif %}23 24{% if tool_available %}25## Available Tool26 27A gutter sizing calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:28 29```bash30python3 /workspace/{{ meta.name }}_calc.py --help31```32 33You may use this tool to verify your calculations or compute values directly.34{% endif %}35 36## Required37 38Calculate the following:39 401. Design stormwater flow Q (L/s)412. Capacity of the smallest adequate standard gutter at the installed grade (L/s)423. Required (smallest adequate) standard gutter size (mm)434. Compliance assessment (1.0 = pass, 0.0 = fail)44 45## Applicable Standards46 47- AS/NZS 3500.3:2025 Plumbing and Drainage — Stormwater Drainage48 49## Constraints50 51- No internet access is available. Work from engineering knowledge and the provided tool.52- Use the following design flow formula (SI units):53 - **Design flow:** Q = I × A / 360054 - where I is rainfall intensity (mm/hr), A is catchment area (m²)55 - Q is in litres per second (L/s)56- Gutter capacities at the reference grade of 1:500 (0.2%) per AS/NZS 3500.3 Table 4.2:57 58| Gutter Profile | Nominal Size (mm) | Capacity at 1:500 (L/s) |59|----------------|-------------------|--------------------------|60| 100mm quad | 100 | 0.6 |61| 115mm quad | 115 | 0.9 |62| 125mm half-round | 125 | 1.0 |63| 150mm quad | 150 | 1.8 |64| 150mm half-round | 150 | 2.0 |65| 175mm OG | 175 | 2.5 |66 67- For grades other than 1:500, capacity scales by √(grade / 0.002), where grade is expressed as a fraction (e.g. 0.2% = 0.002)68- Select the smallest standard gutter whose adjusted capacity meets or exceeds the design flow69- Compliance: pass (1.0) if selected capacity >= design flow, fail (0.0) otherwise70 71## Output Format72 73Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:74 75```json76{77 "design_flow_l_s": <numeric_value>,78 "gutter_capacity_l_s": <numeric_value>,79 "required_gutter_size_mm": <numeric_value>,80 "compliance": <numeric_value>81}82```83 84Write your complete solution to `/workspace/output.md`.85

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Residential house

residential_house

Single or two-storey residential dwelling with pitched roof and standard eaves gutters

sydney-inner-west-housemelbourne-bayside-cottagebrisbane-queenslander
Parameter ranges
roof_catchment_area_m2
30 – 200
gutter_grade_pct
0.15 – 0.5

Terrace row

terrace_row

Terrace or row-house with narrow roof draining to a single gutter run

sydney-paddington-terracemelbourne-fitzroy-row-houseadelaide-north-terrace
Parameter ranges
roof_catchment_area_m2
20 – 80
gutter_grade_pct
0.1 – 0.3

Commercial awning

commercial_awning

Street-front commercial awning or verandah with a short gutter span

brisbane-fortitude-valley-awningperth-cbd-shopfronthobart-waterfront-verandah
Parameter ranges
roof_catchment_area_m2
15 – 120
gutter_grade_pct
0.2 – 0.5

Industrial shed

industrial_shed

Large industrial or warehouse building with long-span metal roof gutters

darwin-industrial-estatetownsville-port-shedcairns-processing-facility
Parameter ranges
roof_catchment_area_m2
300 – 5000
gutter_grade_pct
0.2 – 1

Example task

sydney-inner-west-house-residential-house-preview — hard difficulty, some inputs hidden.

Single or two-storey residential dwelling with pitched roof and standard eaves gutters. sydney-inner-west-house. Required outputs: design_flow_l_s, gutter_capacity_l_s, required_gutter_size_mm, compliance

The model sees

Scenario context and visible inputs.

roof_catchment_area_m2
30 to 200 m²
gutter_profile
100mm_quad
gutter_grade_pct
0.15 to 0.5 %

Executable tool: gutter-sizing_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Rainfall intensity

    rainfall_intensity_mm_hr

Stand-in text in the prompt

The building is {{ archetype.description }} located at {{ archetype.site_context }}

The model must produce

The scored JSON answer schema.

{
  "design_flow_l_s": <number>,
  "gutter_capacity_l_s": <number>,
  "required_gutter_size_mm": <number>,
  "compliance": <number>
}
  • design_flow_l_s · scored within ±3%
  • gutter_capacity_l_s · scored within ±5%
  • required_gutter_size_mm · scored within ±1%
  • compliance · scored within ±1%