LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
mechanicalwith-tool

Gci Calculation

Calculates observed order of accuracy, Richardson extrapolated value, and fine-grid convergence index for a monotonic three-grid study with equal refinement ratio. The template uses a fixed safety factor of 1.25 and requires the three QoI values to show monotonic convergence.

with-tool: The model is given an executable Python calculator script.

Standards

ASME V&V 20-2009

Tags

cfddeterministicgcimechanicalmesh

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

4 inputs

Always given

Included directly in every task prompt.

4

  • Coarse grid value

    coarse_grid_value

    Quantity of interest on the coarse grid

    -1000000 – 1000000

  • Medium grid value

    medium_grid_value

    Quantity of interest on the medium grid

    -1000000 – 1000000

  • Fine grid value

    fine_grid_value

    Quantity of interest on the fine grid

    -1000000 – 1000000

  • Refinement ratio

    refinement_ratio

    Grid refinement ratio between successive grids

    1.01 – 4

Scored outputs

5 outputs

Observed order

observed_order

Observed order of accuracy

Scores if within ±3% of the reference value.

Extrapolated value

extrapolated_value

Richardson extrapolated fine-grid value

Scores if within ±3% of the reference value.

Approximate relative error pct

approximate_relative_error_pct

Approximate relative error between fine and medium grids

Scores if within ±3% of the reference value.

Gci fine pct

gci_fine_pct

Fine-grid convergence index

Scores if within ±3% of the reference value.

Asymptotic range ratio

asymptotic_range_ratio

Asymptotic range ratio

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All parameters given for a drag coefficient convergence study
medium:
All parameters given across convergence study types
hard:
All parameters given for a pressure-drop convergence study

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • gci-calculation_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a senior mechanical engineer specializing in CFD verification.2 3## Task4 5Calculate the grid convergence index for the monotonic three-grid study in {{ site_context }}.6 7## Given8 9| Parameter | Value | Unit |10| --- | ---: | --- |11| Coarse-grid QoI value | {{ coarse_grid_value }} | - |12| Medium-grid QoI value | {{ medium_grid_value }} | - |13| Fine-grid QoI value | {{ fine_grid_value }} | - |14| Refinement ratio | {{ refinement_ratio }} | - |15 16## Constraints17 18- No internet access is available.19- Use equal refinement ratio between coarse, medium, and fine grids.20- Use `p = ln(abs((phi_3 - phi_2) / (phi_2 - phi_1))) / ln(r)`, where `phi_1` is the fine-grid value, `phi_2` is the medium-grid value, and `phi_3` is the coarse-grid value.21- Use `phi_ext = phi_1 + (phi_1 - phi_2) / (r^p - 1)`.22- Use safety factor `Fs = 1.25`.23- Use `GCI_fine = Fs * abs((phi_1 - phi_2) / phi_1) * 100 / (r^p - 1)`.24 25{% if tool_available %}26## Available Tool27 28A calculation tool is available at `/workspace/{{ meta.name }}_calc.py`. Run it with:29 30```bash31python3 /workspace/{{ meta.name }}_calc.py --help32```33 34You may use this tool to verify your calculations or compute values directly.35{% endif %}36 37## Required38 39Calculate:40 41- Observed order of accuracy (`observed_order`)42- Richardson extrapolated value (`extrapolated_value`)43- Approximate relative error (`approximate_relative_error_pct`)44- Fine-grid convergence index (`gci_fine_pct`)45- Asymptotic range ratio (`asymptotic_range_ratio`)46 47## Output Format48 49Show your step-by-step working in Markdown. At the end of your solution, include a JSON block with your final answers in exactly this format:50 51```json52{53 "observed_order": <numeric_value>,54 "extrapolated_value": <numeric_value>,55 "approximate_relative_error_pct": <numeric_value>,56 "gci_fine_pct": <numeric_value>,57 "asymptotic_range_ratio": <numeric_value>58}59```60 61Write your complete solution to `/workspace/output.md`.62

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Drag coefficient

drag_coefficient

Monotonic drag coefficient mesh convergence study

cfd-drag-studyexternal-flow-model
Parameter ranges
coarse_grid_value
0.65 – 0.85
medium_grid_value
0.43 – 0.48
fine_grid_value
0.34 – 0.38
refinement_ratio
1.5 – 2

Pressure drop

pressure_drop

Monotonic pressure-drop mesh convergence study

duct-cfd-modelpipe-loss-cfd-study
Parameter ranges
coarse_grid_value
380 – 520
medium_grid_value
220 – 260
fine_grid_value
170 – 200
refinement_ratio
1.5 – 2

Example task

duct-cfd-model-pressure-drop-preview — hard difficulty, all inputs given.

Monotonic pressure-drop mesh convergence study. duct-cfd-model. Required outputs: observed_order, extrapolated_value, approximate_relative_error_pct, gci_fine_pct, asymptotic_range_ratio

The model sees

Scenario context and visible inputs.

coarse_grid_value
380 to 520
medium_grid_value
220 to 260
fine_grid_value
170 to 200
refinement_ratio
1.5 to 2

Executable tool: gci-calculation_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "observed_order": <number>,
  "extrapolated_value": <number>,
  "approximate_relative_error_pct": <number>,
  "gci_fine_pct": <number>,
  "asymptotic_range_ratio": <number>
}
  • observed_order · scored within ±3%
  • extrapolated_value · scored within ±3%
  • approximate_relative_error_pct · scored within ±3%
  • gci_fine_pct · scored within ±3%
  • asymptotic_range_ratio · scored within ±3%