LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
structuralwith-tool

Rainscreen Drainage Cavity Fire Material Review Package

Calculates source-bound rainscreen cavity and review metrics from a deterministic SSC-09 task-owned source pack. The template combines cavity depth, ventilation/drainage slot margins, material evidence completeness, fire-stop spacing, thermal-break coverage, review comment closure, and an overall synthetic pass score.

with-tool: The model is given an executable Python calculator script.

Standards

fire-stop coordination workflowsmaterial submittal reviewrainscreen drainage workflows

Tags

cavitydeterministicfacadefire-stopmaterialsrainscreenreviewstructural

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

15 inputs

Always given

Included directly in every task prompt.

15
Show 15 inputs

  • Cavity depth

    cavity_depth_mm

    RAINSCREEN-09-DETAIL-05 cavity depth

    38 mm

  • Minimum cavity depth

    minimum_cavity_depth_mm

    RAINSCREEN-09-DETAIL-05 minimum cavity depth

    25 mm

  • Open joint area

    open_joint_area_cm2_m

    CAVITY-09-DRAIN-05 open joint area

    85 cm2/m

  • Required vent area

    required_vent_area_cm2_m

    CAVITY-09-DRAIN-05 required vent area

    60 cm2/m

  • Drainage slot area

    drainage_slot_area_cm2

    CAVITY-09-DRAIN-05 drainage slot area

    140 cm2

  • Required drainage slot area

    required_drainage_slot_area_cm2

    CAVITY-09-DRAIN-05 required drainage slot area

    120 cm2

  • Product documents submitted

    product_documents_submitted

    MAT-09-DATA-05 submitted product documents

    9

  • Product documents required

    product_documents_required

    MAT-09-DATA-05 required product documents

    10

  • Fire stop spacing

    fire_stop_spacing_m

    FIRESTOP-09-SCHED-05 proposed fire-stop spacing

    2.8 m

  • Max fire stop spacing

    max_fire_stop_spacing_m

    FIRESTOP-09-SCHED-05 maximum fire-stop spacing

    3 m

  • Thermal break count

    thermal_break_count

    RAINSCREEN-09-DETAIL-05 thermal break count

    48

  • Bracket count

    bracket_count

    RAINSCREEN-09-DETAIL-05 bracket count

    50

  • Review comments

    review_comments

    REVIEW-09-MAT-05 review comment count

    5

  • Resolved review comments

    resolved_review_comments

    REVIEW-09-MAT-05 resolved review comment count

    5

  • Critical open comments

    critical_open_comments

    REVIEW-09-MAT-05 critical open comment count

    0

Scored outputs

9 outputs

Cavity depth margin

cavity_depth_margin_mm

Cavity depth minus minimum cavity depth

Scores if within ±3% of the reference value.

Vent area margin cm2

vent_area_margin_cm2_m

Open joint area minus required vent area

Scores if within ±3% of the reference value.

Drainage slot margin cm2

drainage_slot_margin_cm2

Drainage slot area minus required drainage slot area

Scores if within ±3% of the reference value.

Material evidence score

material_evidence_score

Submitted product documents divided by required product documents

Scores if within ±0.3% of the reference value.

Fire stop spacing margin

fire_stop_spacing_margin_m

Maximum fire-stop spacing minus proposed fire-stop spacing

Scores if within ±0.3% of the reference value.

Thermal break coverage fraction

thermal_break_coverage_fraction

Thermal break count divided by bracket count

Scores if within ±0.3% of the reference value.

Review resolution fraction

review_resolution_fraction

Resolved review comments divided by review comments

Scores if within ±0.3% of the reference value.

Critical open comments

critical_open_comments

Critical open review comment count

Scores if within ±0.1% of the reference value.

Overall pass score

overall_pass_score

1.0 when rainscreen drainage, fire/material, and review checks pass

Scores if within ±0.1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-09 rainscreen cavity review package
medium:
All source-pack values given for the SSC-09 rainscreen cavity review package
hard:
All source-pack values given for the SSC-09 rainscreen cavity review package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • rainscreen-drainage-cavity-fire-material-review-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a structural/facade engineer checking a task-owned synthetic SSC-09 rainscreen drainage, cavity, fire/material review package.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. Rainscreen cavity drainage, cladding material-submittal, thermal-break, and fire-stop review workflows shape the practice context only; they are not extra data sources for this instance.4 5## Scene6 7- Product family: `SSC-09-LH-05`8- Rainscreen detail: `RAINSCREEN-09-DETAIL-05`9- Cavity drainage note: `CAVITY-09-DRAIN-05`10- Material data register: `MAT-09-DATA-05`11- Fire-stop schedule: `FIRESTOP-09-SCHED-05`12- Review comment register: `REVIEW-09-MAT-05`13- Envelope coordination memo: `MEMO-09-ENVELOPE-05`14 15## Source Values16 17| Item | Value |18|------|-------|19| Cavity depth | {{ cavity_depth_mm }} mm |20| Minimum cavity depth | {{ minimum_cavity_depth_mm }} mm |21| Open joint area | {{ open_joint_area_cm2_m }} cm2/m |22| Required vent area | {{ required_vent_area_cm2_m }} cm2/m |23| Drainage slot area | {{ drainage_slot_area_cm2 }} cm2 |24| Required drainage slot area | {{ required_drainage_slot_area_cm2 }} cm2 |25| Product documents submitted | {{ product_documents_submitted }} |26| Product documents required | {{ product_documents_required }} |27| Fire-stop spacing | {{ fire_stop_spacing_m }} m |28| Maximum fire-stop spacing | {{ max_fire_stop_spacing_m }} m |29| Thermal break count | {{ thermal_break_count }} |30| Bracket count | {{ bracket_count }} |31| Review comments | {{ review_comments }} |32| Resolved review comments | {{ resolved_review_comments }} |33| Critical open comments | {{ critical_open_comments }} |34 35## Checks36 37- Cavity depth margin equals cavity depth minus minimum cavity depth.38- Vent area margin equals open joint area minus required vent area.39- Drainage slot margin equals drainage slot area minus required drainage slot area.40- Material evidence score equals submitted product documents divided by required product documents.41- Fire-stop spacing margin equals maximum spacing minus proposed spacing.42- Thermal break coverage fraction equals thermal break count divided by bracket count.43- Review resolution fraction equals resolved review comments divided by review comments.44- Overall pass score is `1.0` only when drainage/cavity/fire margins are non-negative, material evidence is at least `0.9`, thermal-break coverage is at least `0.95`, all review comments are resolved, and no critical comments remain; otherwise it is `0.0`.45 46## Output Format47 48Write a compact rainscreen review memo to `/workspace/output.md`. Include a source-boundary statement that this is a task-owned synthetic source pack. Preserve the object IDs above and state whether the baseline source pack passes the current docs-only checks.49 50Do not claim authority approval, accepted project evidence, software export validity, full standards compliance, executable real source-pack parsing, generated benchmark readiness, or benchmark readiness.51 52Include a fenced JSON block with exactly these numeric keys:53 54```json55{56 "cavity_depth_margin_mm": <numeric_value>,57 "vent_area_margin_cm2_m": <numeric_value>,58 "drainage_slot_margin_cm2": <numeric_value>,59 "material_evidence_score": <numeric_value>,60 "fire_stop_spacing_margin_m": <numeric_value>,61 "thermal_break_coverage_fraction": <numeric_value>,62 "review_resolution_fraction": <numeric_value>,63 "critical_open_comments": <numeric_value>,64 "overall_pass_score": <numeric_value>65}66```67

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc09 rainscreen cavity review case

ssc09_rainscreen_cavity_review_case

Rainscreen drainage, cavity, fire/material review case

Parameter ranges
cavity_depth_mm
38
minimum_cavity_depth_mm
25
open_joint_area_cm2_m
85
required_vent_area_cm2_m
60
drainage_slot_area_cm2
140
required_drainage_slot_area_cm2
120
product_documents_submitted
9
product_documents_required
10
fire_stop_spacing_m
2.8
max_fire_stop_spacing_m
3
thermal_break_count
48
bracket_count
50
review_comments
5
resolved_review_comments
5
critical_open_comments
0

Example task

ssc09-rainscreen-cavity-review-case-preview — hard difficulty, all inputs given.

Rainscreen drainage, cavity, fire/material review case. Required outputs: cavity_depth_margin_mm, vent_area_margin_cm2_m, drainage_slot_margin_cm2, material_evidence_score, fire_stop_spacing_margin_m, thermal_break_coverage_fraction

The model sees

Scenario context and visible inputs.

cavity_depth_mm
38 to 38 mm
minimum_cavity_depth_mm
25 to 25 mm
open_joint_area_cm2_m
85 to 85 cm2/m
required_vent_area_cm2_m
60 to 60 cm2/m
drainage_slot_area_cm2
140 to 140 cm2
required_drainage_slot_area_cm2
120 to 120 cm2
product_documents_submitted
9 to 9
product_documents_required
10 to 10
fire_stop_spacing_m
2.8 to 2.8 m
max_fire_stop_spacing_m
3 to 3 m
thermal_break_count
48 to 48
bracket_count
50 to 50
review_comments
5 to 5
resolved_review_comments
5 to 5
critical_open_comments
0 to 0

Executable tool: rainscreen-drainage-cavity-fire-material-review-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "cavity_depth_margin_mm": <number>,
  "vent_area_margin_cm2_m": <number>,
  "drainage_slot_margin_cm2": <number>,
  "material_evidence_score": <number>,
  "fire_stop_spacing_margin_m": <number>,
  "thermal_break_coverage_fraction": <number>,
  "review_resolution_fraction": <number>,
  "critical_open_comments": <number>,
  "overall_pass_score": <number>
}
  • cavity_depth_margin_mm · scored within ±3%
  • vent_area_margin_cm2_m · scored within ±3%
  • drainage_slot_margin_cm2 · scored within ±3%
  • material_evidence_score · scored within ±0.3%
  • fire_stop_spacing_margin_m · scored within ±0.3%
  • thermal_break_coverage_fraction · scored within ±0.3%
  • review_resolution_fraction · scored within ±0.3%
  • critical_open_comments · scored within ±0.1%
  • overall_pass_score · scored within ±0.1%