LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
mechanicalwith-tool

Treatment Review Permit Basis Package

Calculates source-bound treatment review metrics from a deterministic SSC-10 task-owned source pack. The template combines SRT, permit margins, process capacity margins, comment closure, source completeness, and an overall synthetic pass score.

with-tool: The model is given an executable Python calculator script.

Standards

nitrification SRT workflowpermit review workflowprocess calculation appendix workflow

Tags

chemicaldeterministicmechanicaloxygenpermitreviewsludgesrtwastewater

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

19 inputs

Always given

Included directly in every task prompt.

19
Show 19 inputs

  • Base required srt

    base_required_srt_d

    CALC-10-APPX-08 base required SRT

    4.5 d

  • Temperature c

    temperature_c

    SAMPLE-10-DATA-08 governing temperature

    14 deg C

  • Theta factor

    theta_factor

    CALC-10-APPX-08 temperature correction factor

    1.07

  • Actual srt

    actual_srt_d

    CALC-10-APPX-08 actual SRT

    9.2 d

  • Permit bod

    permit_bod_mg_l

    PERMIT-10-CRIT-08 permit BOD limit

    25 mg/L

  • Predicted bod

    predicted_bod_mg_l

    SAMPLE-10-DATA-08 predicted BOD

    18 mg/L

  • Permit ammonia

    permit_ammonia_mg_l

    PERMIT-10-CRIT-08 permit ammonia limit

    3 mg/L

  • Predicted ammonia

    predicted_ammonia_mg_l

    SAMPLE-10-DATA-08 predicted ammonia

    1.9 mg/L

  • Oxygen required

    oxygen_required_kg_d

    CALC-10-APPX-08 oxygen required

    840 kg/d

  • Oxygen capacity

    oxygen_capacity_kg_d

    CALC-10-APPX-08 oxygen capacity

    965 kg/d

  • Chemical required

    chemical_required_kg_d

    CALC-10-APPX-08 chemical required

    182 kg/d

  • Chemical capacity

    chemical_capacity_kg_d

    CALC-10-APPX-08 chemical capacity

    220 kg/d

  • Sludge predicted

    sludge_predicted_kg_d

    CALC-10-APPX-08 sludge predicted

    910 kg/d

  • Sludge handling capacity

    sludge_handling_capacity_kg_d

    CALC-10-APPX-08 sludge handling capacity

    1040 kg/d

  • Total review comments

    total_review_comments

    COMMENT-10-REG-08 total review comments

    18

  • Closed review comments

    closed_review_comments

    COMMENT-10-REG-08 closed review comments

    16

  • Critical comments open

    critical_comments_open

    COMMENT-10-REG-08 critical comments still open

    0

  • Source references found

    source_references_found

    AUTH-10-MATRIX-08 source references found

    13

  • Source references required

    source_references_required

    AUTH-10-MATRIX-08 source references required

    14

Scored outputs

12 outputs

Required srt d

required_srt_d

Temperature-adjusted required SRT

Scores if within ±3% of the reference value.

Srt margin d

srt_margin_d

Actual SRT minus required SRT

Scores if within ±3% of the reference value.

Bod permit margin mg l

bod_permit_margin_mg_l

BOD permit limit minus predicted BOD

Scores if within ±3% of the reference value.

Ammonia permit margin mg l

ammonia_permit_margin_mg_l

Ammonia permit limit minus predicted ammonia

Scores if within ±3% of the reference value.

Oxygen capacity margin kg d

oxygen_capacity_margin_kg_d

Oxygen capacity minus oxygen required

Scores if within ±3% of the reference value.

Chemical capacity margin kg d

chemical_capacity_margin_kg_d

Chemical capacity minus chemical required

Scores if within ±3% of the reference value.

Sludge capacity margin kg d

sludge_capacity_margin_kg_d

Sludge handling capacity minus predicted sludge

Scores if within ±3% of the reference value.

Comments resolved fraction

comments_resolved_fraction

Closed review comments divided by total comments

Scores if within ±3% of the reference value.

Source completeness fraction

source_completeness_fraction

Found source references divided by required references

Scores if within ±3% of the reference value.

Response completeness score

response_completeness_score

Average response completeness score

Scores if within ±3% of the reference value.

Critical comments open

critical_comments_open

Critical review comments still open

Scores if within ±0.1% of the reference value.

Overall pass score

overall_pass_score

1.0 when permit, process capacity, and critical-comment checks pass

Scores if within ±0.1% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

All inputs remain visible at every tier

For this template, difficulty scales through parameter and scenario ranges rather than hidden information.

easy:
All source-pack values given for the SSC-10 permit review package
medium:
All source-pack values given for the SSC-10 permit review package
hard:
All source-pack values given for the SSC-10 permit review package

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • treatment-review-permit-basis-package_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1You are a wastewater process engineer checking a task-owned synthetic SSC-10 treatment review response and permit-basis package.2 3Use only the task-owned synthetic source pack values shown below for numeric grading. Permit criteria, sampling data, process calculation appendix, comment register, and authority-matrix workflows shape the context only; this instance does not parse a real permit, model branch, or accepted project report.4 5## Scene6 7- Product: `SSC-10-LH-08`8- Permit criteria table: `PERMIT-10-CRIT-08`9- Sampling dataset: `SAMPLE-10-DATA-08`10- Process calculation appendix: `CALC-10-APPX-08`11- Comment register: `COMMENT-10-REG-08`12- Authority matrix: `AUTH-10-MATRIX-08`13- Compliance gap memo: `MEMO-10-PERMIT-08`14 15## Source Values16 17| Item | Value |18| --- | --- |19| Base required SRT | {{ base_required_srt_d }} d |20| Governing temperature | {{ temperature_c }} deg C |21| Theta factor | {{ theta_factor }} |22| Actual SRT | {{ actual_srt_d }} d |23| Permit BOD | {{ permit_bod_mg_l }} mg/L |24| Predicted BOD | {{ predicted_bod_mg_l }} mg/L |25| Permit ammonia | {{ permit_ammonia_mg_l }} mg/L |26| Predicted ammonia | {{ predicted_ammonia_mg_l }} mg/L |27| Oxygen required | {{ oxygen_required_kg_d }} kg/d |28| Oxygen capacity | {{ oxygen_capacity_kg_d }} kg/d |29| Chemical required | {{ chemical_required_kg_d }} kg/d |30| Chemical capacity | {{ chemical_capacity_kg_d }} kg/d |31| Sludge predicted | {{ sludge_predicted_kg_d }} kg/d |32| Sludge handling capacity | {{ sludge_handling_capacity_kg_d }} kg/d |33| Review comments | {{ total_review_comments }} total |34| Closed comments | {{ closed_review_comments }} |35| Critical comments open | {{ critical_comments_open }} |36| Source references found | {{ source_references_found }} |37| Source references required | {{ source_references_required }} |38 39## Calculation Rules40 41- Required SRT equals `base_required_srt_d x theta_factor^(20 - temperature_c)`.42- SRT margin equals actual SRT minus required SRT.43- Permit margins equal permit limits minus predicted values.44- Process capacity margins equal listed capacity minus required or predicted values.45- Comments resolved fraction equals closed review comments divided by total review comments.46- Source completeness fraction equals found source references divided by required references.47- Response completeness score averages comment closure, source completeness, and critical-comment closure.48- Overall pass score is `1.0` only when permit, process capacity, and critical-comment checks pass.49 50## Output Format51 52Write a compact memo to `/workspace/output.md`. Include a source-boundary statement that this is a task-owned synthetic source pack. Preserve the source IDs above and state whether the baseline source pack passes the current docs-only checks.53 54Do not claim authority approval, accepted project evidence, software export validity, full standards compliance, executable real source-pack parsing, generated modelling evidence, generated benchmark readiness, or benchmark readiness.55 56Include a fenced JSON block with exactly these numeric keys:57 58```json59{60 "required_srt_d": <numeric_value>,61 "srt_margin_d": <numeric_value>,62 "bod_permit_margin_mg_l": <numeric_value>,63 "ammonia_permit_margin_mg_l": <numeric_value>,64 "oxygen_capacity_margin_kg_d": <numeric_value>,65 "chemical_capacity_margin_kg_d": <numeric_value>,66 "sludge_capacity_margin_kg_d": <numeric_value>,67 "comments_resolved_fraction": <numeric_value>,68 "source_completeness_fraction": <numeric_value>,69 "response_completeness_score": <numeric_value>,70 "critical_comments_open": <numeric_value>,71 "overall_pass_score": <numeric_value>72}73```74

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Ssc10 treatment permit review case

ssc10_treatment_permit_review_case

Treatment review response and permit-basis source pack

treatment-permit-reviewpermit-response-margin-check
Parameter ranges
params
{'base_required_srt_d': {'min': 4.5, 'max': 4.5}, 'temperature_c': {'min': 14.0, 'max': 14.0}, 'theta_factor': {'min': 1.07, 'max': 1.07}, 'actual_srt_d': {'min': 9.2, 'max': 9.2}, 'permit_bod_mg_l': {'min': 25.0, 'max': 25.0}, 'predicted_bod_mg_l': {'min': 18.0, 'max': 18.0}, 'permit_ammonia_mg_l': {'min': 3.0, 'max': 3.0}, 'predicted_ammonia_mg_l': {'min': 1.9, 'max': 1.9}, 'oxygen_required_kg_d': {'min': 840.0, 'max': 840.0}, 'oxygen_capacity_kg_d': {'min': 965.0, 'max': 965.0}, 'chemical_required_kg_d': {'min': 182.0, 'max': 182.0}, 'chemical_capacity_kg_d': {'min': 220.0, 'max': 220.0}, 'sludge_predicted_kg_d': {'min': 910.0, 'max': 910.0}, 'sludge_handling_capacity_kg_d': {'min': 1040.0, 'max': 1040.0}, 'total_review_comments': {'min': 18.0, 'max': 18.0}, 'closed_review_comments': {'min': 16.0, 'max': 16.0}, 'critical_comments_open': {'min': 0.0, 'max': 0.0}, 'source_references_found': {'min': 13.0, 'max': 13.0}, 'source_references_required': {'min': 14.0, 'max': 14.0}}

Example task

treatment-permit-review-ssc10-treatment-permit-review-case-preview — hard difficulty, all inputs given.

Treatment review response and permit-basis source pack. treatment-permit-review. Required outputs: required_srt_d, srt_margin_d, bod_permit_margin_mg_l, ammonia_permit_margin_mg_l, oxygen_capacity_margin_kg_d, chemical_capacity_margin_kg_d

The model sees

Scenario context and visible inputs.

base_required_srt_d
4.5 to 4.5 d
temperature_c
14 to 14 deg C
theta_factor
1.07 to 1.07
actual_srt_d
9.2 to 9.2 d
permit_bod_mg_l
25 to 25 mg/L
predicted_bod_mg_l
18 to 18 mg/L
permit_ammonia_mg_l
3 to 3 mg/L
predicted_ammonia_mg_l
1.9 to 1.9 mg/L
oxygen_required_kg_d
840 to 840 kg/d
oxygen_capacity_kg_d
965 to 965 kg/d
chemical_required_kg_d
182 to 182 kg/d
chemical_capacity_kg_d
220 to 220 kg/d
sludge_predicted_kg_d
910 to 910 kg/d
sludge_handling_capacity_kg_d
1040 to 1040 kg/d
total_review_comments
18 to 18
closed_review_comments
16 to 16
critical_comments_open
0 to 0
source_references_found
13 to 13
source_references_required
14 to 14

Executable tool: treatment-review-permit-basis-package_calc.py

The model must infer

Inputs withheld at this difficulty.

Nothing. All inputs are supplied.

The model must produce

The scored JSON answer schema.

{
  "required_srt_d": <number>,
  "srt_margin_d": <number>,
  "bod_permit_margin_mg_l": <number>,
  "ammonia_permit_margin_mg_l": <number>,
  "oxygen_capacity_margin_kg_d": <number>,
  "chemical_capacity_margin_kg_d": <number>,
  "sludge_capacity_margin_kg_d": <number>,
  "comments_resolved_fraction": <number>,
  "source_completeness_fraction": <number>,
  "response_completeness_score": <number>,
  "critical_comments_open": <number>,
  "overall_pass_score": <number>
}
  • required_srt_d · scored within ±3%
  • srt_margin_d · scored within ±3%
  • bod_permit_margin_mg_l · scored within ±3%
  • ammonia_permit_margin_mg_l · scored within ±3%
  • oxygen_capacity_margin_kg_d · scored within ±3%
  • chemical_capacity_margin_kg_d · scored within ±3%
  • sludge_capacity_margin_kg_d · scored within ±3%
  • comments_resolved_fraction · scored within ±3%
  • source_completeness_fraction · scored within ±3%
  • response_completeness_score · scored within ±3%
  • critical_comments_open · scored within ±0.1%
  • overall_pass_score · scored within ±0.1%