LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
structuralwith-tool

Steel Certificate Structural Fire Carbon Package

Calculates source-bound steel certificate compliance metrics from a deterministic SSC-15 task-owned source pack. The template combines one mill certificate, material schedule, welding criterion, fire note, structural excerpt, and material compliance memo.

with-tool: The model is given an executable Python calculator script.

Standards

IIW carbon equivalentfire design note reviewstructural material certificate review

Tags

carbon-equivalentcertificatedeterministicfireproduct-compliancesteelstructural

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

16 inputs

Always given

Included directly in every task prompt.

16
Show 16 inputs

  • Carbon percent

    carbon_percent

    CERT-15-STEEL-01 carbon content from the mill certificate

    0.18 %

  • Manganese percent

    manganese_percent

    CERT-15-STEEL-01 manganese content from the mill certificate

    1.25 %

  • Chromium percent

    chromium_percent

    CERT-15-STEEL-01 chromium content from the mill certificate

    0.25 %

  • Molybdenum percent

    molybdenum_percent

    CERT-15-STEEL-01 molybdenum content from the mill certificate

    0.04 %

  • Vanadium percent

    vanadium_percent

    CERT-15-STEEL-01 vanadium content from the mill certificate

    0.03 %

  • Nickel percent

    nickel_percent

    CERT-15-STEEL-01 nickel content from the mill certificate

    0.2 %

  • Copper percent

    copper_percent

    CERT-15-STEEL-01 copper content from the mill certificate

    0.25 %

  • Carbon equivalent limit

    carbon_equivalent_limit

    WELD-15-CRIT-01 project carbon-equivalent acceptance limit

    0.52

  • Certificate capacity

    certificate_capacity_kn

    MAT-15-SCHED-01 certificate-derived member capacity

    128 kN

  • Design load

    design_load_kn

    CALC-15-STRUCT-01 governing design load consumed by the certificate check

    95 kN

  • Fire load ratio

    fire_load_ratio

    FIRE-15-NOTE-01 structural fire load ratio used by the task-owned critical-temperature formula

    0.62

  • Required fire temperature c

    required_fire_temperature_c

    FIRE-15-NOTE-01 required steel temperature threshold

    430 degC

  • Matching certificate fields

    matching_certificate_fields

    CERT-15-STEEL-01 certificate fields matching the material schedule

    6 count

  • Required certificate fields

    required_certificate_fields

    MAT-15-SCHED-01 required certificate field count

    6 count

  • Completed memo sections

    completed_memo_sections

    MEMO-15-STEEL-01 completed material memo sections

    5 count

  • Required memo sections

    required_memo_sections

    MEMO-15-STEEL-01 required material memo sections

    5 count

Scored outputs

9 outputs

Carbon equivalent

carbon_equivalent

IIW-style carbon equivalent from certificate chemistry

Scores if within ±0.3% of the reference value.

Carbon equivalent margin

carbon_equivalent_margin

Carbon equivalent limit minus calculated carbon equivalent

Scores if within ±0.3% of the reference value.

Structural capacity margin kn

structural_capacity_margin_kn

Certificate capacity minus design load

Scores if within ±3% of the reference value.

Structural utilization

structural_utilization

Design load divided by certificate capacity

Scores if within ±0.3% of the reference value.

Critical steel temperature c

critical_steel_temperature_c

Task-owned critical steel temperature from fire load ratio

Scores if within ±3% of the reference value.

Fire temperature margin c

fire_temperature_margin_c

Critical steel temperature minus required fire temperature

Scores if within ±3% of the reference value.

Certificate field match fraction

certificate_field_match_fraction

Matching certificate fields divided by required certificate fields

Scores if within ±0.3% of the reference value.

Material memo completeness fraction

material_memo_completeness_fraction

Completed memo sections divided by required memo sections

Scores if within ±0.3% of the reference value.

Overall pass score

overall_pass_score

1.0 when certificate, weldability, structural, fire, and memo checks pass

Scores if within ±0.1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-15 steel certificate compliance package
medium:
All source-pack values given for the SSC-15 steel certificate compliance package
hard:
All source-pack values given for the SSC-15 steel certificate compliance package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • steel-certificate-structural-fire-carbon-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a structural material reviewer checking a task-owned synthetic SSC-15 steel certificate package.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. Product compliance directories and certificate workflows shape the context only; they are not extra data sources for this instance.4 5## Scene6 7- Product: `SSC-15-LH-01`8- Mill certificate: `CERT-15-STEEL-01`9- Material schedule: `MAT-15-SCHED-01`10- Welding criterion: `WELD-15-CRIT-01`11- Fire design note: `FIRE-15-NOTE-01`12- Structural calculation excerpt: `CALC-15-STRUCT-01`13- Material compliance memo: `MEMO-15-STEEL-01`14 15All checks use the same steel certificate and material schedule. Do not change the product identity, certificate identity, material grade, fire basis, or memo status unless you explicitly flag a source conflict.16 17## Source Values18 19| Item | Value |20|------|-------|21| Carbon | {{ carbon_percent }} percent |22| Manganese | {{ manganese_percent }} percent |23| Chromium | {{ chromium_percent }} percent |24| Molybdenum | {{ molybdenum_percent }} percent |25| Vanadium | {{ vanadium_percent }} percent |26| Nickel | {{ nickel_percent }} percent |27| Copper | {{ copper_percent }} percent |28| Carbon-equivalent limit | {{ carbon_equivalent_limit }} |29| Certificate capacity | {{ certificate_capacity_kn }} kN |30| Design load | {{ design_load_kn }} kN |31| Fire load ratio | {{ fire_load_ratio }} |32| Required fire temperature | {{ required_fire_temperature_c }} degC |33| Matching certificate fields | {{ matching_certificate_fields }} |34| Required certificate fields | {{ required_certificate_fields }} |35| Completed memo sections | {{ completed_memo_sections }} |36| Required memo sections | {{ required_memo_sections }} |37 38Carbon equivalent equals `C + Mn/6 + (Cr + Mo + V)/5 + (Ni + Cu)/15`. Critical steel temperature equals `905 - 690 x fire_load_ratio`.39 40## Output Format41 42Write a compact material compliance memo to `/workspace/output.md`. Include a source-boundary statement that this is a task-owned synthetic source pack. Do not claim authority approval, accepted project evidence, software export validity, full standards compliance, executable real source-pack parsing, generated benchmark readiness, or benchmark readiness.43 44Include a fenced JSON block with exactly these numeric keys:45 46```json47{48 "carbon_equivalent": <numeric_value>,49 "carbon_equivalent_margin": <numeric_value>,50 "structural_capacity_margin_kn": <numeric_value>,51 "structural_utilization": <numeric_value>,52 "critical_steel_temperature_c": <numeric_value>,53 "fire_temperature_margin_c": <numeric_value>,54 "certificate_field_match_fraction": <numeric_value>,55 "material_memo_completeness_fraction": <numeric_value>,56 "overall_pass_score": <numeric_value>57}58```59

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc15 steel certificate case

ssc15_steel_certificate_case

Steel certificate structural, fire, and carbon compliance case

ssc15-steel-certificate
Parameter ranges

Example task

ssc15-steel-certificate-ssc15-steel-certificate-case-preview — hard difficulty, all inputs given.

Steel certificate structural, fire, and carbon compliance case. ssc15-steel-certificate. Required outputs: carbon_equivalent, carbon_equivalent_margin, structural_capacity_margin_kn, structural_utilization, critical_steel_temperature_c, fire_temperature_margin_c

The model sees

Scenario context and visible inputs.

carbon_percent
0.18 to 0.18 %
manganese_percent
1.25 to 1.25 %
chromium_percent
0.25 to 0.25 %
molybdenum_percent
0.04 to 0.04 %
vanadium_percent
0.03 to 0.03 %
nickel_percent
0.2 to 0.2 %
copper_percent
0.25 to 0.25 %
carbon_equivalent_limit
0.52 to 0.52
certificate_capacity_kn
128 to 128 kN
design_load_kn
95 to 95 kN
fire_load_ratio
0.62 to 0.62
required_fire_temperature_c
430 to 430 degC
matching_certificate_fields
6 to 6 count
required_certificate_fields
6 to 6 count
completed_memo_sections
5 to 5 count
required_memo_sections
5 to 5 count

Executable tool: steel-certificate-structural-fire-carbon-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "carbon_equivalent": <number>,
  "carbon_equivalent_margin": <number>,
  "structural_capacity_margin_kn": <number>,
  "structural_utilization": <number>,
  "critical_steel_temperature_c": <number>,
  "fire_temperature_margin_c": <number>,
  "certificate_field_match_fraction": <number>,
  "material_memo_completeness_fraction": <number>,
  "overall_pass_score": <number>
}
  • carbon_equivalent · scored within ±0.3%
  • carbon_equivalent_margin · scored within ±0.3%
  • structural_capacity_margin_kn · scored within ±3%
  • structural_utilization · scored within ±0.3%
  • critical_steel_temperature_c · scored within ±3%
  • fire_temperature_margin_c · scored within ±3%
  • certificate_field_match_fraction · scored within ±0.3%
  • material_memo_completeness_fraction · scored within ±0.3%
  • overall_pass_score · scored within ±0.1%