LIVEdataset aec-bench@releasetasks 552models 18disciplines 5last submission … · built …

groundno-tool

Ground Structural Electrical Issue Review Package

Review-first source-packet task for a geotechnical/electrical handoff: inventory source files, preserve geotechnical versus resistivity authority, recompute bearing and earthing checks from source-owned methods, and issue an auditable readiness decision.

no-tool: The model must reason numerically unaided.

Standards

IEEE 80-2013 Terzaghi 1943 task-owned synthetic criteria

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

01
Template
The reusable contract shown on this page.
02
Scenario
An archetype and site context are sampled.
03
Difficulty tier
Inputs may be hidden at harder tiers.
04
Task prompt
The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

19 inputs

Always given

Included directly in every task prompt.

Show 14 inputs

Raw n value
raw_n_value
Raw SPT N value from SPT-07 record
18 – 26
Effective overburden
effective_overburden_kpa
Effective overburden at the SPT depth
112 – 145 kPa
Design groundwater level
design_groundwater_level_m
Design groundwater depth below ground level
1.45 – 2.35 m
Cohesion
cohesion_kpa
Effective cohesion for bearing check
4 – 8 kPa
Total unit weight
total_unit_weight_kn_m3
Total soil unit weight
18 – 19.6 kN/m3
Footing width
footing_width_m
Square footing width
2.2 – 2.8 m
Embedment depth
embedment_depth_m
Footing embedment depth
1 – 1.35 m
Factor of safety
factor_of_safety
Source-owned bearing factor of safety
3
Soil resistivity
soil_resistivity_ohm_m
Apparent resistivity from RES-07 traverse
65 – 105 ohm-m
Grid length
grid_length_m
Earthing grid length
28 – 38 m
Grid width
grid_width_m
Earthing grid width
20 – 28 m
Total conductor length
total_conductor_length_m
Total buried grid conductor length
320 – 430 m
Burial depth
burial_depth_m
Grid burial depth
0.5 – 0.75 m
Grid current
grid_current_ka
Maximum grid current
2.8 – 3.8 kA

Hidden at higher difficulty

Visible in easier tasks and withheld in one or more harder tiers.

Bearing margin target
bearing_margin_target_kpa
Hidden clean-case target bearing margin used to derive applied pressure
Hidden at easy and medium and hard difficulty.
35 – 70 kPa
Bearing deficit
bearing_deficit_kpa
Hidden deficient-case bearing deficit
Hidden at easy and medium and hard difficulty.
18 – 42 kPa
Grid resistance margin target
grid_resistance_margin_target_ohm
Hidden clean-case grid resistance margin used to derive the limit
Hidden at easy and medium and hard difficulty.
0.12 – 0.28 ohm
Touch voltage margin target
touch_voltage_margin_target_v
Hidden clean-case touch-voltage margin used to derive the limit
Hidden at easy and medium and hard difficulty.
120 – 260 V
Packet variant
packet_variant
Hidden source-packet issue variant
Hidden at easy and medium and hard difficulty.
cleanmissing_groundwater_levelstale_ground_memo_revisionresistivity_strength_misusescenario_copy_forwardopen_critical_commentminor_open_comment_carriedbearing_fos_deficient

Scored outputs

20 outputs

Rlr 01 status

rlr_01_status

RLR-01 status code

Scores if within ±1% of the reference value.

Rlr 02 status

rlr_02_status

RLR-02 status code

Scores if within ±1% of the reference value.

Rlr 03 status

rlr_03_status

RLR-03 status code

Scores if within ±1% of the reference value.

Rlr 04 status

rlr_04_status

RLR-04 status code

Scores if within ±1% of the reference value.

Rlr 05 status

rlr_05_status

RLR-05 status code

Scores if within ±1% of the reference value.

Rlr 06 status

rlr_06_status

RLR-06 status code

Scores if within ±1% of the reference value.

Rlr 07 status

rlr_07_status

RLR-07 status code

Scores if within ±1% of the reference value.

Rlr 08 status

rlr_08_status

RLR-08 status code

Scores if within ±1% of the reference value.

Rlr 09 status

rlr_09_status

RLR-09 status code

Scores if within ±1% of the reference value.

Corrected spt n60

corrected_spt_n60

Energy, borehole, sampler, and rod-length corrected SPT N60

Scores if within ±3% of the reference value.

Design friction angle deg

design_friction_angle_deg

Source-owned design friction angle from SPT N60 correlation

Scores if within ±3% of the reference value.

Allowable bearing

allowable_bearing_kpa

Allowable Terzaghi bearing capacity

Scores if within ±3% of the reference value.

Bearing margin

bearing_margin_kpa

Allowable bearing pressure minus applied bearing pressure

Scores if within ±3% of the reference value.

Grid resistance ohm

grid_resistance_ohm

Earthing grid resistance

Scores if within ±3% of the reference value.

Grid resistance margin ohm

grid_resistance_margin_ohm

Grid resistance limit minus grid resistance

Scores if within ±3% of the reference value.

Touch voltage margin

touch_voltage_margin_v

Touch voltage limit minus source-owned touch voltage

Scores if within ±3% of the reference value.

Readiness code

readiness_code

Readiness decision code

Scores if within ±1% of the reference value.

Required findings count

required_findings_count

Required finding count

Scores if within ±1% of the reference value.

Required information requests count

required_information_requests_count

Required information request count

Scores if within ±1% of the reference value.

Required carried actions count

required_carried_actions_count

Required carried action count

Scores if within ±1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

Some inputs hidden

Obvious packet defects: clean, missing groundwater evidence, or bearing failure

Hidden inputs

Packet variantpacket_variant
Bearing margin targetbearing_margin_target_kpa
Bearing deficitbearing_deficit_kpa
Grid resistance margin target ohmgrid_resistance_margin_target_ohm
Touch voltage margin targettouch_voltage_margin_target_v

Vary restricted to: raw_n_value, design_groundwater_level_m, soil_resistivity_ohm_m, packet_variant

Packet variant restricted to: clean, missing_groundwater_level, bearing_fos_deficient

medium

Some inputs hidden

Full packet-variant distribution

Hidden inputs

Packet variantpacket_variant
Bearing margin targetbearing_margin_target_kpa
Bearing deficitbearing_deficit_kpa
Grid resistance margin target ohmgrid_resistance_margin_target_ohm
Touch voltage margin targettouch_voltage_margin_target_v

Vary restricted to: raw_n_value, effective_overburden_kpa, design_groundwater_level_m, cohesion_kpa, total_unit_weight_kn_m3, footing_width_m, embedment_depth_m, bearing_margin_target_kpa, bearing_deficit_kpa, soil_resistivity_ohm_m, grid_length_m, grid_width_m, total_conductor_length_m, grid_current_ka, grid_resistance_margin_target_ohm, touch_voltage_margin_target_v, packet_variant

hard

Some inputs hidden

Subtle source-authority, stale-revision, scenario-copy, and comment-closure defects

Hidden inputs

Packet variantpacket_variant
Bearing margin targetbearing_margin_target_kpa
Bearing deficitbearing_deficit_kpa
Grid resistance margin target ohmgrid_resistance_margin_target_ohm
Touch voltage margin targettouch_voltage_margin_target_v

Vary restricted to: raw_n_value, effective_overburden_kpa, design_groundwater_level_m, cohesion_kpa, total_unit_weight_kn_m3, footing_width_m, embedment_depth_m, bearing_margin_target_kpa, bearing_deficit_kpa, soil_resistivity_ohm_m, grid_length_m, grid_width_m, total_conductor_length_m, burial_depth_m, grid_current_ka, grid_resistance_margin_target_ohm, touch_voltage_margin_target_v, packet_variant

Packet variant restricted to: clean, missing_groundwater_level, stale_ground_memo_revision, resistivity_strength_misuse, scenario_copy_forward, open_critical_comment, minor_open_comment_carried, bearing_fos_deficient

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

instruction.md

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are the independent reviewing engineer for a ground structural-electrical issue package covering one site, borehole/SPT logs, groundwater record, ground interpretation memo, foundation load table, resistivity survey, earthing grid design, and criteria/comments memo.2 3A source packet has been placed in `/workspace/sources/`. It contains a document register, borehole/SPT logs, groundwater record, ground interpretation memo, foundation load table, resistivity survey, earthing grid design extract, and criteria memo with review comments. The packet is a task-owned synthetic source pack; treat it as the only source of numeric truth for this review.4 5Your job is not to redesign the foundation or earthing grid. Your job is to decide whether the package is ready to issue, and to produce an auditable review record.6 7## Review Workflow8 91. Inventory the source packet before drawing any conclusion. Record every document ID, revision, and status.102. Build an identity ledger: site, borehole, SPT record, groundwater record, ground memo, foundation, resistivity survey, earthing grid, load case, and criteria memo.113. Check for source conflicts, stale revisions, copied scenarios, missing evidence, authority-partition errors between ground strength and resistivity, and open critical comments before accepting any package claim.124. Recompute the package's own calculations only where they answer review items, using the assessment bases stated in the criteria memo. Do not import methods or values from outside the packet.135. Assign exactly one status to every review item: `pass`, `fail`, `not_applicable`, or `insufficient_data`.146. Do not invent missing values. Mark missing evidence as `insufficient_data` and request the exact missing field and source. A value that a source explicitly marks as pending or awaiting confirmation is missing evidence of this kind: it does not make otherwise-reconciling identifiers inconsistent and does not make evidence that is present untraceable; the check that cannot be completed without it takes `insufficient_data`.157. Convert every failure into a finding with a source pointer, affected object, consequence, and corrective action.168. Issue a readiness decision that reconciles with your matrix, findings, information requests, and action register.17 18## Review Matrix19 20Assess each item and give it exactly one status:21 22| Item | Review question |23|---|---|24| RLR-01 | Packet completeness: are all required borehole/SPT, groundwater, ground interpretation, foundation load, resistivity, earthing grid, and criteria files present with IDs and revisions? |25| RLR-02 | Object identity and authority partition: do the site, borehole, SPT, groundwater, ground memo, foundation, resistivity survey, earthing grid, and criteria memo stay consistent without using resistivity evidence as strength evidence? |26| RLR-03 | Ground interpretation basis: are corrected SPT, friction-angle selection, bearing parameters, groundwater, and earthing inputs traceable, current, and recomputable? |27| RLR-04 | Structural bearing adequacy: does allowable bearing capacity clear the applied foundation load using the source-owned water-table correction? |28| RLR-05 | Scenario consequence: is the same structure, load case, and design basis used across the ground memo, foundation table, and earthing grid package? |29| RLR-06 | Electrical earthing resilience: are grid resistance and touch-voltage margins source-backed and internally consistent with the resistivity survey and grid design extract? |30| RLR-07 | Comment and action closure: is every critical comment closed, and are carried minor comments owner/action controlled? |31| RLR-08 | Readiness consistency: does your final decision match your own matrix, findings, information requests, and action register? |32| RLR-09 | Claim boundary: does your review avoid unsupported approval, compliance, source-hardening, executable-verifier, or benchmark-readiness claims? |33 34## Boundary Rules35 36- Use the review matrix definitions to decide the most specific affected item from the source packet. Do not infer a status from this instruction alone.37- If a source value needed for a recomputation is absent, omit the dependent `computed_evidence` key and raise an information request for the missing field and source. Do not include missing or unrecomputable keys with `null`, `0`, or placeholder values.38- Assign additional failures only when the source packet gives independent evidence for them; do not double-count one source issue across unrelated matrix rows.39- RLR-08 passes when the readiness decision reconciles with the review matrix, findings, information requests, and action register.40- Every finding, information request, and action must name one exact RLR item. Use a single RLR item per register row; do not write combined items such as `RLR-04/RLR-06`.41- Do not rename computed_evidence keys.42 43## Output44 45Write your complete review to `/workspace/output.md`. Explain your reasoning briefly in prose, then end with exactly one fenced JSON block:46 47```json48{49  "source_inventory": [{"doc_id": "...", "revision": "...", "status": "..."}],50  "identity_ledger": {51    "site": "...",52    "borehole": "...",53    "spt_record": "...",54    "groundwater_record": "...",55    "ground_memo": "...",56    "foundation": "...",57    "resistivity_survey": "...",58    "earthing_grid": "...",59    "load_case": "...",60    "criteria_memo": "..."61  },62  "review_matrix": {63    "RLR-01": {"status": "pass|fail|not_applicable|insufficient_data", "evidence": "..."},64    "RLR-02": {"status": "...", "evidence": "..."},65    "RLR-03": {"status": "...", "evidence": "..."},66    "RLR-04": {"status": "...", "evidence": "..."},67    "RLR-05": {"status": "...", "evidence": "..."},68    "RLR-06": {"status": "...", "evidence": "..."},69    "RLR-07": {"status": "...", "evidence": "..."},70    "RLR-08": {"status": "...", "evidence": "..."},71    "RLR-09": {"status": "...", "evidence": "..."}72  },73  "computed_evidence": {74    "corrected_spt_n60": 0.0,75    "design_friction_angle_deg": 0.0,76    "allowable_bearing_kpa": 0.0,77    "bearing_margin_kpa": 0.0,78    "grid_resistance_ohm": 0.0,79    "grid_resistance_margin_ohm": 0.0,80    "touch_voltage_margin_v": 0.081  },82  "findings": [83    {"item": "RLR-0X", "severity": "critical|minor", "source_id": "...", "object_id": "...", "consequence": "...", "action": "..."}84  ],85  "information_requests": [86    {"item": "RLR-0X", "missing_field": "...", "source_id": "..."}87  ],88  "action_register": [89    {"action": "...", "owner": "...", "linked_item": "RLR-0X"}90  ],91  "readiness_decision": "ready_to_issue|ready_with_carried_actions|not_ready_to_issue",92  "claim_boundary_statement": "..."93}94```95 96Rules for the structured block:97 98- `computed_evidence` values must come from your own recomputation from packet source values. Omit a key only when its inputs are missing from the packet, then raise the matching information request instead. Do not include missing or unrecomputable keys with `null`, `0`, or placeholder values.99- Every `fail` needs at least one finding with a non-empty `source_id`, `object_id`, `consequence`, and `action`.100- Every `insufficient_data` needs an information request naming the exact missing field and its source document.101- Every `not_applicable` needs a scope reason in its matrix `evidence`.102- Carried actions must appear in `action_register` with an owner.103- `readiness_decision` must reconcile with your matrix: unresolved failures or missing critical evidence mean the package is not ready.104- `claim_boundary_statement` must state that this review covers a task-owned synthetic source packet and does not claim authority approval, accepted project evidence, full standards compliance, source-pack hardening, executable-verifier readiness, or benchmark readiness.105

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Transport equipment site

transport_equipment_site

Transport corridor equipment site with shallow square foundation and moderate earthing grid

transport-equipment-padrail-corridor-ground-grid

Parameter ranges

raw_n_value: 18 – 23
design_groundwater_level_m: 1.45 – 2.1
soil_resistivity_ohm_m: 65 – 92

Coastal utility site

coastal_utility_site

Coastal utility site with slightly denser sand and larger earthing grid

coastal-utility-padsubstation-edge-foundation

Parameter ranges

raw_n_value: 21 – 26
design_groundwater_level_m: 1.8 – 2.35
soil_resistivity_ohm_m: 78 – 105

Example task

transport-equipment-pad-transport-equipment-site-preview — hard difficulty, some inputs hidden.

Transport corridor equipment site with shallow square foundation and moderate earthing grid. transport-equipment-pad. Required outputs: rlr_01_status, rlr_02_status, rlr_03_status, rlr_04_status, rlr_05_status, rlr_06_status

The model sees

Scenario context and visible inputs.

raw_n_value: 18 to 23
effective_overburden_kpa: 112 to 145 kPa
design_groundwater_level_m: 1.45 to 2.1 m
cohesion_kpa: 4 to 8 kPa
total_unit_weight_kn_m3: 18 to 19.6 kN/m3
footing_width_m: 2.2 to 2.8 m
embedment_depth_m: 1 to 1.35 m
factor_of_safety: 3 to 3
soil_resistivity_ohm_m: 65 to 92 ohm-m
grid_length_m: 28 to 38 m
grid_width_m: 20 to 28 m
total_conductor_length_m: 320 to 430 m
burial_depth_m: 0.5 to 0.75 m
grid_current_ka: 2.8 to 3.8 kA

Executable tool: ground-structural-electrical-issue-review-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Touch voltage margin target
touch_voltage_margin_target_v
Packet variant
packet_variant
Grid resistance margin target ohm
grid_resistance_margin_target_ohm
Bearing margin target
bearing_margin_target_kpa
Bearing deficit
bearing_deficit_kpa

The model must produce

The scored JSON answer schema.

{
  "rlr_01_status": <number>,
  "rlr_02_status": <number>,
  "rlr_03_status": <number>,
  "rlr_04_status": <number>,
  "rlr_05_status": <number>,
  "rlr_06_status": <number>,
  "rlr_07_status": <number>,
  "rlr_08_status": <number>,
  "rlr_09_status": <number>,
  "corrected_spt_n60": <number>,
  "design_friction_angle_deg": <number>,
  "allowable_bearing_kpa": <number>,
  "bearing_margin_kpa": <number>,
  "grid_resistance_ohm": <number>,
  "grid_resistance_margin_ohm": <number>,
  "touch_voltage_margin_v": <number>,
  "readiness_code": <number>,
  "required_findings_count": <number>,
  "required_information_requests_count": <number>,
  "required_carried_actions_count": <number>
}

rlr_01_status · scored within ±1%
rlr_02_status · scored within ±1%
rlr_03_status · scored within ±1%
rlr_04_status · scored within ±1%
rlr_05_status · scored within ±1%
rlr_06_status · scored within ±1%
rlr_07_status · scored within ±1%
rlr_08_status · scored within ±1%
rlr_09_status · scored within ±1%
corrected_spt_n60 · scored within ±3%
design_friction_angle_deg · scored within ±3%
allowable_bearing_kpa · scored within ±3%
bearing_margin_kpa · scored within ±3%
grid_resistance_ohm · scored within ±3%
grid_resistance_margin_ohm · scored within ±3%
touch_voltage_margin_v · scored within ±3%
readiness_code · scored within ±1%
required_findings_count · scored within ±1%
required_information_requests_count · scored within ±1%
required_carried_actions_count · scored within ±1%

Ground Structural Electrical Issue Review Package

How this task is generated

Template

Scenario

Difficulty tier

Task prompt

Parameters

Inputs

Scored outputs

Rlr 01 status

Rlr 02 status

Rlr 03 status

Rlr 04 status

Rlr 05 status

Rlr 06 status

Rlr 07 status

Rlr 08 status

Rlr 09 status

Corrected spt n60

Design friction angle deg

Allowable bearing

Bearing margin

Grid resistance ohm

Grid resistance margin ohm

Touch voltage margin

Readiness code

Required findings count

Required information requests count

Required carried actions count

Difficulty

easy

medium

hard

Task bundle

Scenario archetypes

Transport equipment site

Coastal utility site

Example task

The model sees

The model must infer

The model must produce