LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
civilboth

Freeboard Calculation

Determines the total freeboard and minimum crest or floor level for coastal and flood-prone structures by summing component allowances for wave overtopping, sea level rise, construction tolerance, and a consequence-based safety margin, per NZS 4404:2010 and MfE Guidance 2024. Used in coastal and floodplain development to set minimum building levels above the design water surface.

both: The model must reason numerically unaided.

Standards

MfE Guidance 2024NZS 4404:2010

Tags

coastaldrainagefloodfreeboardsea-level-rise

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

5 inputs

Always given

Included directly in every task prompt.

1

  • Design water level

    design_water_level_m

    Design still-water level including tidal and storm surge components (m above datum)

    0.5 – 8 m

Derived from scenario

Sampled from the scenario and inferable from its description.

4

  • Wave allowance

    wave_allowance_m

    Wave overtopping allowance depending on wave height and structure type

    Derived from the archetype scenario. Hidden at hard difficulty.

    0.3 – 1 m

  • Slr allowance

    slr_allowance_m

    Climate change sea level rise allowance for planning horizon

    Derived from the archetype scenario. Hidden at hard difficulty.

    0.1 – 1 m

  • Construction tolerance

    construction_tolerance_m

    Construction tolerance allowance

    Derived from the archetype scenario. Hidden at hard difficulty.

    0.05 – 0.15 m

  • Safety margin

    safety_margin_m

    Safety margin based on consequence category

    Derived from the archetype scenario. Hidden at hard difficulty.

    0.15 – 0.5 m

Scored outputs

2 outputs

Total freeboard

total_freeboard_m

Total freeboard allowance (m)

Scores if within ±3% of the reference value.

Minimum crest level

minimum_crest_level_m

Minimum crest or floor level (m above datum)

Scores if within ±5% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Low consequence structure, all parameters given — straightforward addition

medium

All inputs given

Higher consequence, larger allowances — all parameters given, same formula

hard

Some inputs hidden

Wave allowance, SLR allowance, construction tolerance, and safety margin hidden — agent must infer from structure type and consequence category

Hidden inputs

  • Wave allowancewave_allowance_m
  • Slr allowanceslr_allowance_m
  • Construction toleranceconstruction_tolerance_m
  • Safety marginsafety_margin_m

Prompt replacement text

The structure is {{ archetype.description }} ({{ archetype.site_context }})

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior coastal/flood engineer specializing in freeboard design for coastal and flood-prone structures.2 3## Problem4 5Calculate the required total freeboard allowance and minimum crest (or floor) level for a coastal or flood structure.6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Design water level (DWL) | {{ design_water_level_m }} | m above datum |12{% if wave_allowance_m is defined %}13| Wave overtopping allowance | {{ wave_allowance_m }} | m |14{% endif %}15{% if slr_allowance_m is defined %}16| Sea level rise allowance | {{ slr_allowance_m }} | m |17{% endif %}18{% if construction_tolerance_m is defined %}19| Construction tolerance | {{ construction_tolerance_m }} | m |20{% endif %}21{% if safety_margin_m is defined %}22| Safety margin | {{ safety_margin_m }} | m |23{% endif %}24{% if archetype_description is defined %}25 26### Structure and Site Conditions27 28{{ archetype_description }}29{% endif %}30 31{% if tool_available %}32## Available Tool33 34A freeboard calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:35 36```bash37python3 /workspace/{{ meta.name }}_calc.py --help38```39 40You may use this tool to verify your calculations or compute values directly.41{% endif %}42 43## Required44 45Calculate the following:46 471. Total freeboard allowance (m) — the sum of all component allowances482. Minimum crest or floor level (m above datum) — design water level plus total freeboard49 50## Applicable Standards51 52- NZS 4404:2010 (Land Development and Subdivision Infrastructure)53- MfE Guidance 2024 (Coastal Hazards and Climate Change)54 55## Constraints56 57- No internet access is available. Work from engineering knowledge and the provided tool.58- Use the component-based freeboard approach:59 - Total freeboard = wave_allowance + slr_allowance + construction_tolerance + safety_margin60 - Minimum crest level = design_water_level + total_freeboard61- Wave overtopping allowance depends on wave height and structure type (typical range 0.3–1.0 m)62- Sea level rise allowance depends on planning horizon (typical range 0.1–1.0 m)63- Construction tolerance is typically 0.05–0.15 m64- Safety margin depends on consequence category (typical range 0.15–0.50 m)65 66## Output Format67 68Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:69 70```json71{72 "total_freeboard_m": <numeric_value>,73 "minimum_crest_level_m": <numeric_value>74}75```76 77Write your complete solution to `/workspace/output.md`.78

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Residential coastal

residential_coastal

Residential coastal dwelling with low consequence category

gold-coast-beachfront-houseauckland-harbour-dwellingchristchurch-estuary-residence
Parameter ranges
wave_allowance_m
0.3 – 0.5
slr_allowance_m
0.3 – 0.6
construction_tolerance_m
0.05 – 0.1
safety_margin_m
0.15 – 0.25

Commercial waterfront

commercial_waterfront

Commercial waterfront development with medium consequence category

sydney-darling-harbour-precinctwellington-waterfront-retailbrisbane-river-commercial
Parameter ranges
wave_allowance_m
0.4 – 0.7
slr_allowance_m
0.4 – 0.8
construction_tolerance_m
0.08 – 0.12
safety_margin_m
0.25 – 0.35

Critical infrastructure

critical_infrastructure

Critical infrastructure with high consequence category (hospital, power station)

tauranga-port-substationdarwin-hospital-coastaltownsville-water-treatment-plant
Parameter ranges
wave_allowance_m
0.6 – 1
slr_allowance_m
0.6 – 1
construction_tolerance_m
0.1 – 0.15
safety_margin_m
0.35 – 0.5

Seawall revetment

seawall_revetment

Seawall or revetment structure protecting coastal assets

napier-marine-parade-seawallcairns-esplanade-revetmentperth-scarborough-seawall
Parameter ranges
wave_allowance_m
0.5 – 0.8
slr_allowance_m
0.4 – 0.9
construction_tolerance_m
0.08 – 0.12
safety_margin_m
0.2 – 0.4

Example task

gold-coast-beachfront-house-residential-coastal-preview — hard difficulty, some inputs hidden.

Residential coastal dwelling with low consequence category. gold-coast-beachfront-house. Required outputs: total_freeboard_m, minimum_crest_level_m

The model sees

Scenario context and visible inputs.

design_water_level_m
0.5 to 8 m

Executable tool: freeboard-calculation_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Construction tolerance

    construction_tolerance_m

  • Safety margin

    safety_margin_m

  • Slr allowance

    slr_allowance_m

  • Wave allowance

    wave_allowance_m

Stand-in text in the prompt

The structure is {{ archetype.description }} ({{ archetype.site_context }})

The model must produce

The scored JSON answer schema.

{
  "total_freeboard_m": <number>,
  "minimum_crest_level_m": <number>
}
  • total_freeboard_m · scored within ±3%
  • minimum_crest_level_m · scored within ±5%