LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
civilwith-tool

Concrete Mix Drainage Retaining Compliance Package

Calculates source-bound concrete or mix compliance metrics from a deterministic SSC-15 task-owned source pack. The template combines one mix design, SCM product record, strength criterion, foundation/retaining detail, exposure note, and mix compliance memo.

with-tool: The model is given an executable Python calculator script.

Standards

SCM substitution reviewfoundation and exposure criteriamix design review

Tags

bearingcertificatecivilconcretedeterministicdrainagemix-designproduct-compliance

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

14 inputs

Always given

Included directly in every task prompt.

14
Show 14 inputs

  • Specified strength

    specified_strength_mpa

    STRENGTH-15-CRIT-03 specified concrete strength

    40 MPa

  • Submitted mean strength

    submitted_mean_strength_mpa

    MIX-15-DESIGN-03 submitted mean mix strength

    48 MPa

  • Standard deviation

    standard_deviation_mpa

    MIX-15-DESIGN-03 strength standard deviation

    4 MPa

  • Target strength factor

    target_strength_factor

    STRENGTH-15-CRIT-03 target mean strength factor

    1.65

  • Scm replacement percent

    scm_replacement_percent

    SCM-15-PRODUCT-03 proposed SCM replacement

    25 %

  • Max scm replacement percent

    max_scm_replacement_percent

    SCM-15-PRODUCT-03 maximum accepted SCM replacement

    35 %

  • Bearing capacity

    bearing_capacity_kpa

    FOUND-15-DETAIL-03 bearing capacity for the use case

    340 kPa

  • Bearing demand

    bearing_demand_kpa

    FOUND-15-DETAIL-03 bearing demand consumed by the mix memo

    260 kPa

  • Exposure cover provided

    exposure_cover_provided_mm

    EXP-15-NOTE-03 provided cover for exposure condition

    55 mm

  • Exposure cover required

    exposure_cover_required_mm

    EXP-15-NOTE-03 required cover for exposure condition

    45 mm

  • Drainage freeboard provided

    drainage_freeboard_provided_m

    FOUND-15-DETAIL-03 provided drainage freeboard

    0.35 m

  • Drainage freeboard required

    drainage_freeboard_required_m

    EXP-15-NOTE-03 required drainage freeboard

    0.25 m

  • Matching evidence items

    matching_evidence_items

    MEMO-15-MIX-03 evidence items matched across source files

    6 count

  • Required evidence items

    required_evidence_items

    MEMO-15-MIX-03 required evidence item count

    6 count

Scored outputs

9 outputs

Target mean strength mpa

target_mean_strength_mpa

Specified strength plus target factor times standard deviation

Scores if within ±3% of the reference value.

Strength margin mpa

strength_margin_mpa

Submitted mean strength minus target mean strength

Scores if within ±3% of the reference value.

Scm replacement margin

scm_replacement_margin_percent

Maximum SCM replacement minus proposed SCM replacement

Scores if within ±3% of the reference value.

Bearing capacity margin

bearing_capacity_margin_kpa

Bearing capacity minus bearing demand

Scores if within ±3% of the reference value.

Bearing utilization

bearing_utilization

Bearing demand divided by bearing capacity

Scores if within ±0.3% of the reference value.

Exposure cover margin

exposure_cover_margin_mm

Provided cover minus required cover

Scores if within ±3% of the reference value.

Drainage freeboard margin

drainage_freeboard_margin_m

Provided drainage freeboard minus required drainage freeboard

Scores if within ±0.3% of the reference value.

Mix evidence match fraction

mix_evidence_match_fraction

Matching evidence items divided by required evidence items

Scores if within ±0.3% of the reference value.

Overall pass score

overall_pass_score

1.0 when strength, SCM, bearing, exposure, drainage, and evidence checks pass

Scores if within ±0.1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-15 concrete mix compliance package
medium:
All source-pack values given for the SSC-15 concrete mix compliance package
hard:
All source-pack values given for the SSC-15 concrete mix compliance package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • concrete-mix-drainage-retaining-compliance-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a civil materials reviewer checking a task-owned synthetic SSC-15 concrete or mix compliance package.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. Product certificate and mix-design workflows shape the context only; they are not extra data sources for this instance.4 5## Scene6 7- Product: `SSC-15-LH-03`8- Mix design sheet: `MIX-15-DESIGN-03`9- SCM product data: `SCM-15-PRODUCT-03`10- Strength criterion: `STRENGTH-15-CRIT-03`11- Foundation/retaining detail: `FOUND-15-DETAIL-03`12- Exposure class note: `EXP-15-NOTE-03`13- Mix compliance memo: `MEMO-15-MIX-03`14 15All checks use the same mix design, product selection, and use-case boundary. Do not change the mix identity, SCM record, foundation demand, or exposure basis unless you explicitly flag a source conflict.16 17## Source Values18 19| Item | Value |20|------|-------|21| Specified strength | {{ specified_strength_mpa }} MPa |22| Submitted mean strength | {{ submitted_mean_strength_mpa }} MPa |23| Standard deviation | {{ standard_deviation_mpa }} MPa |24| Target strength factor | {{ target_strength_factor }} |25| SCM replacement | {{ scm_replacement_percent }} percent |26| Maximum SCM replacement | {{ max_scm_replacement_percent }} percent |27| Bearing capacity | {{ bearing_capacity_kpa }} kPa |28| Bearing demand | {{ bearing_demand_kpa }} kPa |29| Exposure cover provided | {{ exposure_cover_provided_mm }} mm |30| Exposure cover required | {{ exposure_cover_required_mm }} mm |31| Drainage freeboard provided | {{ drainage_freeboard_provided_m }} m |32| Drainage freeboard required | {{ drainage_freeboard_required_m }} m |33| Matching evidence items | {{ matching_evidence_items }} |34| Required evidence items | {{ required_evidence_items }} |35 36Target mean strength equals specified strength plus target factor times standard deviation.37 38## Output Format39 40Write a compact mix compliance memo to `/workspace/output.md`. Include a source-boundary statement that this is a task-owned synthetic source pack. Do not claim authority approval, accepted project evidence, software export validity, full standards compliance, executable real source-pack parsing, generated benchmark readiness, or benchmark readiness.41 42Include a fenced JSON block with exactly these numeric keys:43 44```json45{46 "target_mean_strength_mpa": <numeric_value>,47 "strength_margin_mpa": <numeric_value>,48 "scm_replacement_margin_percent": <numeric_value>,49 "bearing_capacity_margin_kpa": <numeric_value>,50 "bearing_utilization": <numeric_value>,51 "exposure_cover_margin_mm": <numeric_value>,52 "drainage_freeboard_margin_m": <numeric_value>,53 "mix_evidence_match_fraction": <numeric_value>,54 "overall_pass_score": <numeric_value>55}56```57

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc15 concrete mix case

ssc15_concrete_mix_case

Concrete mix drainage and retaining compliance case

ssc15-concrete-mix
Parameter ranges

Example task

ssc15-concrete-mix-ssc15-concrete-mix-case-preview — hard difficulty, all inputs given.

Concrete mix drainage and retaining compliance case. ssc15-concrete-mix. Required outputs: target_mean_strength_mpa, strength_margin_mpa, scm_replacement_margin_percent, bearing_capacity_margin_kpa, bearing_utilization, exposure_cover_margin_mm

The model sees

Scenario context and visible inputs.

specified_strength_mpa
40 to 40 MPa
submitted_mean_strength_mpa
48 to 48 MPa
standard_deviation_mpa
4 to 4 MPa
target_strength_factor
1.65 to 1.65
scm_replacement_percent
25 to 25 %
max_scm_replacement_percent
35 to 35 %
bearing_capacity_kpa
340 to 340 kPa
bearing_demand_kpa
260 to 260 kPa
exposure_cover_provided_mm
55 to 55 mm
exposure_cover_required_mm
45 to 45 mm
drainage_freeboard_provided_m
0.35 to 0.35 m
drainage_freeboard_required_m
0.25 to 0.25 m
matching_evidence_items
6 to 6 count
required_evidence_items
6 to 6 count

Executable tool: concrete-mix-drainage-retaining-compliance-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "target_mean_strength_mpa": <number>,
  "strength_margin_mpa": <number>,
  "scm_replacement_margin_percent": <number>,
  "bearing_capacity_margin_kpa": <number>,
  "bearing_utilization": <number>,
  "exposure_cover_margin_mm": <number>,
  "drainage_freeboard_margin_m": <number>,
  "mix_evidence_match_fraction": <number>,
  "overall_pass_score": <number>
}
  • target_mean_strength_mpa · scored within ±3%
  • strength_margin_mpa · scored within ±3%
  • scm_replacement_margin_percent · scored within ±3%
  • bearing_capacity_margin_kpa · scored within ±3%
  • bearing_utilization · scored within ±0.3%
  • exposure_cover_margin_mm · scored within ±3%
  • drainage_freeboard_margin_m · scored within ±0.3%
  • mix_evidence_match_fraction · scored within ±0.3%
  • overall_pass_score · scored within ±0.1%