LIVEdataset aec-bench@releasetasks 552models 18disciplines 5last submission … · built …

structuralwith-tool

Carbon Equivalent Calc

Calculates the International Institute of Welding carbon equivalent from steel chemistry using CE = C + Mn/6 + (Cr + Mo + V)/5 + (Ni + Cu)/15. The template reports numeric margins to weldability thresholds and a deterministic preheat indicator for first-pass steel specification checks.

with-tool: The model is given an executable Python calculator script.

Standards

AS/NZS 1554 AWS D1.1 IIW formula

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

01
Template
The reusable contract shown on this page.
02
Scenario
An archetype and site context are sampled.
03
Difficulty tier
Inputs may be hidden at harder tiers.
04
Task prompt
The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

9 inputs

Always given

Included directly in every task prompt.

Show 9 inputs

Carbon pct
carbon_pct
Carbon content C
0.02 – 0.35 %
Manganese pct
manganese_pct
Manganese content Mn
0.2 – 1.8 %
Chromium pct
chromium_pct
Chromium content Cr
0 – 1.5 %
Molybdenum pct
molybdenum_pct
Molybdenum content Mo
0 – 0.6 %
Vanadium pct
vanadium_pct
Vanadium content V
0 – 0.25 %
Nickel pct
nickel_pct
Nickel content Ni
0 – 2 %
Copper pct
copper_pct
Copper content Cu
0 – 0.8 %
Caution threshold pct
caution_threshold_pct
Carbon equivalent threshold where preheat review is indicated
0.3 – 0.5 %
High risk threshold pct
high_risk_threshold_pct
Carbon equivalent threshold for high weldability risk
0.45 – 0.7 %

Scored outputs

5 outputs

Carbon equivalent pct

carbon_equivalent_pct

IIW carbon equivalent CE

Scores if within ±3% of the reference value.

Caution margin pct

caution_margin_pct

Margin above the caution threshold

Scores if within ±3% of the reference value.

High risk margin pct

high_risk_margin_pct

Margin above the high-risk threshold

Scores if within ±3% of the reference value.

Weldability risk index

weldability_risk_index

Numeric weldability risk class: 0 low, 1 caution, 2 high

Scores if within ±1% of the reference value.

Preheat indicated

preheat_indicated

Numeric preheat indication: 0 no, 1 yes

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:: All chemistry and thresholds given for low-carbon steel
medium:: All chemistry and thresholds given across common structural steels
hard:: All chemistry and thresholds given for higher-risk repair welding cases

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

instruction.md
carbon-equivalent-calc_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior structural engineer specializing in steel specification and welding.2 3## Problem4 5Calculate the steel carbon equivalent using the IIW formula and determine numeric weldability risk indicators.6 7## Given8 9| Parameter | Value | Unit |10|-----------|-------|------|11| Carbon C | {{ carbon_pct }} | % |12| Manganese Mn | {{ manganese_pct }} | % |13| Chromium Cr | {{ chromium_pct }} | % |14| Molybdenum Mo | {{ molybdenum_pct }} | % |15| Vanadium V | {{ vanadium_pct }} | % |16| Nickel Ni | {{ nickel_pct }} | % |17| Copper Cu | {{ copper_pct }} | % |18| Caution threshold | {{ caution_threshold_pct }} | % |19| High-risk threshold | {{ high_risk_threshold_pct }} | % |20 21{% if archetype_description is defined %}22### Steel Context23 24{{ archetype_description }}25{% endif %}26 27{% if tool_available %}28## Available Tool29 30A carbon equivalent calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:31 32```bash33python3 /workspace/{{ meta.name }}_calc.py --help34```35 36You may use this tool to verify your calculations or compute values directly.37{% endif %}38 39## Required40 41Calculate the following:42 431. Carbon equivalent CE (%)442. Margin above the caution threshold (%)453. Margin above the high-risk threshold (%)464. Numeric weldability risk class: 0 low, 1 caution, 2 high475. Numeric preheat indication: 0 no, 1 yes48 49## Constraints50 51- No internet access is available. Work from engineering knowledge and the provided tool.52- Use CE = C + Mn/6 + (Cr + Mo + V)/5 + (Ni + Cu)/15.53- Use risk class 0 when CE is below the caution threshold.54- Use risk class 1 when CE is at or above the caution threshold and below the high-risk threshold.55- Use risk class 2 when CE is at or above the high-risk threshold.56- Set preheat indicated to 1 when CE is at or above the caution threshold, otherwise 0.57 58## Output Format59 60Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:61 62```json63{64  "carbon_equivalent_pct": <numeric_value>,65  "caution_margin_pct": <numeric_value>,66  "high_risk_margin_pct": <numeric_value>,67  "weldability_risk_index": <numeric_value>,68  "preheat_indicated": <numeric_value>69}70```71 72Write your complete solution to `/workspace/output.md`.73

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Low carbon plate

low_carbon_plate

Low-carbon structural plate with low alloy content

building-frame-steelbridge-secondary-steel

Parameter ranges

carbon_pct: 0.08 – 0.16
manganese_pct: 0.6 – 1.2
chromium_pct: 0 – 0.2
molybdenum_pct: 0 – 0.08
vanadium_pct: 0 – 0.05
nickel_pct: 0 – 0.25
copper_pct: 0 – 0.25
caution_threshold_pct: 0.38 – 0.42
high_risk_threshold_pct: 0.55 – 0.6

Hsla plate

hsla_plate

High-strength low-alloy plate with moderate alloy content

crane-runway-girderheavy-industrial-platform

Parameter ranges

carbon_pct: 0.12 – 0.22
manganese_pct: 1 – 1.7
chromium_pct: 0.1 – 0.6
molybdenum_pct: 0.02 – 0.25
vanadium_pct: 0.02 – 0.12
nickel_pct: 0.1 – 0.8
copper_pct: 0.05 – 0.35
caution_threshold_pct: 0.38 – 0.45
high_risk_threshold_pct: 0.55 – 0.65

Repair welding unknown origin

repair_welding_unknown_origin

Repair welding material with elevated alloy content

brownfield-plant-repairmarine-structure-repair

Parameter ranges

carbon_pct: 0.18 – 0.3
manganese_pct: 1.2 – 1.8
chromium_pct: 0.3 – 1.2
molybdenum_pct: 0.05 – 0.4
vanadium_pct: 0.02 – 0.18
nickel_pct: 0.2 – 1.5
copper_pct: 0.1 – 0.6
caution_threshold_pct: 0.38 – 0.45
high_risk_threshold_pct: 0.55 – 0.7

Example task

crane-runway-girder-hsla-plate-preview — hard difficulty, all inputs given.

High-strength low-alloy plate with moderate alloy content. crane-runway-girder. Required outputs: carbon_equivalent_pct, caution_margin_pct, high_risk_margin_pct, weldability_risk_index, preheat_indicated

The model sees

Scenario context and visible inputs.

carbon_pct: 0.12 to 0.22 %
manganese_pct: 1 to 1.7 %
chromium_pct: 0.1 to 0.6 %
molybdenum_pct: 0.02 to 0.25 %
vanadium_pct: 0.02 to 0.12 %
nickel_pct: 0.1 to 0.8 %
copper_pct: 0.05 to 0.35 %
caution_threshold_pct: 0.38 to 0.45 %
high_risk_threshold_pct: 0.55 to 0.65 %

Executable tool: carbon-equivalent-calc_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "carbon_equivalent_pct": <number>,
  "caution_margin_pct": <number>,
  "high_risk_margin_pct": <number>,
  "weldability_risk_index": <number>,
  "preheat_indicated": <number>
}

carbon_equivalent_pct · scored within ±3%
caution_margin_pct · scored within ±3%
high_risk_margin_pct · scored within ±3%
weldability_risk_index · scored within ±1%
preheat_indicated · scored within ±1%

Carbon Equivalent Calc

How this task is generated

Template

Scenario

Difficulty tier

Task prompt

Parameters

Inputs

Scored outputs

Carbon equivalent pct

Caution margin pct

High risk margin pct

Weldability risk index

Preheat indicated

Difficulty

All inputs remain visible at every tier

Task bundle

Scenario archetypes

Low carbon plate

Hsla plate

Repair welding unknown origin

Example task

The model sees

The model must infer

The model must produce