LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
structuralwith-tool

Construction Tolerance

Sums fabrication, erection, survey, movement, and clearance allowances for a deterministic construction tolerance check. The template also reports root-sum-square tolerance for the first four components and calculates the required slot length using the total allowance at both slot ends.

with-tool: The model is given an executable Python calculator script.

Standards

AISC 303AS 4100EN 1090-2

Tags

constructiondeterministicfit-upstructuraltolerance

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

6 inputs

Always given

Included directly in every task prompt.

6

  • Fabrication tolerance

    fabrication_tolerance_mm

    Fabrication tolerance component

    0 – 50 mm

  • Erection tolerance

    erection_tolerance_mm

    Erection tolerance component

    0 – 75 mm

  • Survey tolerance

    survey_tolerance_mm

    Survey tolerance component

    0 – 30 mm

  • Movement allowance

    movement_allowance_mm

    Movement allowance component

    0 – 100 mm

  • Clearance

    clearance_mm

    Installation clearance allowance

    0 – 50 mm

  • Component length

    component_length_mm

    Nominal component length accommodated by the slot

    50 – 5000 mm

Scored outputs

4 outputs

Total allowance

total_allowance_mm

Arithmetic sum of tolerance and clearance components

Scores if within ±3% of the reference value.

Rss tolerance

rss_tolerance_mm

Root-sum-square tolerance from fabrication, erection, survey, and movement components

Scores if within ±3% of the reference value.

Required slot length

required_slot_length_mm

Slot length required for the component and allowance at both ends

Scores if within ±3% of the reference value.

Clearance included

clearance_included_mm

Clearance allowance included in the total allowance

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All parameters given for a steel bracket slot
medium:
All parameters given across bracket and facade slot checks
hard:
All parameters given for a facade panel slot

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • construction-tolerance_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior structural engineer specializing in construction tolerances and fit-up allowances.2 3## Problem4 5Calculate the construction tolerance allowance for a slotted connection and determine the required slot length.6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Fabrication tolerance | {{ fabrication_tolerance_mm }} | mm |12| Erection tolerance | {{ erection_tolerance_mm }} | mm |13| Survey tolerance | {{ survey_tolerance_mm }} | mm |14| Movement allowance | {{ movement_allowance_mm }} | mm |15| Clearance allowance | {{ clearance_mm }} | mm |16| Component length | {{ component_length_mm }} | mm |17 18{% if archetype_description is defined %}19### Structural Context20 21{{ archetype_description }}22{% endif %}23 24{% if tool_available %}25## Available Tool26 27A construction tolerance calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:28 29```bash30python3 /workspace/{{ meta.name }}_calc.py --help31```32 33You may use this tool to verify your calculations or compute values directly.34{% endif %}35 36## Required37 38Calculate the following:39 401. Total allowance as fabrication tolerance plus erection tolerance plus survey tolerance plus movement allowance plus clearance allowance412. RSS tolerance as the square root of the sum of squares of fabrication, erection, survey, and movement components only423. Required slot length as component length plus twice the total allowance434. Clearance allowance included in the total allowance44 45## Constraints46 47- No internet access is available. Work from engineering knowledge and the provided tool.48- Use only the provided tolerance components.49- Do not include clearance in the RSS tolerance.50- Use the total allowance on both ends of the slot when calculating required slot length.51 52## Output Format53 54Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:55 56```json57{58 "total_allowance_mm": <numeric_value>,59 "rss_tolerance_mm": <numeric_value>,60 "required_slot_length_mm": <numeric_value>,61 "clearance_included_mm": <numeric_value>62}63```64 65Write your complete solution to `/workspace/output.md`.66

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Steel bracket slot

steel_bracket_slot

Steel bracket slotted connection tolerance check

steel-frameindustrial-platform
Parameter ranges
fabrication_tolerance_mm
2 – 12
erection_tolerance_mm
5 – 25
survey_tolerance_mm
2 – 10
movement_allowance_mm
2 – 20
clearance_mm
2 – 12
component_length_mm
100 – 600

Facade panel slot

facade_panel_slot

Facade panel slotted fixing tolerance check

facade-supportbuilding-envelope
Parameter ranges
fabrication_tolerance_mm
3 – 20
erection_tolerance_mm
10 – 40
survey_tolerance_mm
3 – 15
movement_allowance_mm
10 – 60
clearance_mm
5 – 20
component_length_mm
300 – 2000

Example task

facade-support-facade-panel-slot-preview — hard difficulty, all inputs given.

Facade panel slotted fixing tolerance check. facade-support. Required outputs: total_allowance_mm, rss_tolerance_mm, required_slot_length_mm, clearance_included_mm

The model sees

Scenario context and visible inputs.

fabrication_tolerance_mm
3 to 20 mm
erection_tolerance_mm
10 to 40 mm
survey_tolerance_mm
3 to 15 mm
movement_allowance_mm
10 to 60 mm
clearance_mm
5 to 20 mm
component_length_mm
300 to 2000 mm

Executable tool: construction-tolerance_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "total_allowance_mm": <number>,
  "rss_tolerance_mm": <number>,
  "required_slot_length_mm": <number>,
  "clearance_included_mm": <number>
}
  • total_allowance_mm · scored within ±3%
  • rss_tolerance_mm · scored within ±3%
  • required_slot_length_mm · scored within ±3%
  • clearance_included_mm · scored within ±3%