LIVEdataset aec-bench@releasetasks 552models 18last submission … · built …
electricalwith-tool

Leni Calculation

Calculates annual lighting energy, Lighting Energy Numeric Indicator, and percentage saving against a reference LENI for an interior lighting zone.

with-tool: The model is given an executable Python calculator script.

Standards

EN 15193-1

Tags

deterministicelectricalenergyinterior-lightinglenilighting

How this task is generated

One template produces many comparable benchmark tasks while keeping the scoring contract fixed.

  1. 01

    Template

    The reusable contract shown on this page.

  2. 02

    Scenario

    An archetype and site context are sampled.

  3. 03

    Difficulty tier

    Inputs may be hidden at harder tiers.

  4. 04

    Task prompt

    The model responds with the declared outputs.

Parameters

Inputs the model receives, and the outputs it is scored on.

Inputs

6 inputs

Always given

Included directly in every task prompt.

5

  • Installed lighting power

    installed_lighting_power_w

    Installed lighting power in the zone

    100 – 200000 W

  • Annual operating hours

    annual_operating_hours

    Annual lighting operating hours

    100 – 8760 h/year

  • Control factor

    control_factor

    Lighting control factor

    0 – 1

  • Zone area

    zone_area_m2

    Interior lighting zone area

    10 – 50000 m2

  • Reference leni

    reference_leni_kwh_m2_year

    Reference LENI used for comparison

    1 – 100 kWh/m2/year

Hidden at higher difficulty

Visible in easier tasks and withheld in one or more harder tiers.

1

  • Daylight factor

    daylight_factor

    Daylight availability factor

    Hidden at hard difficulty.

    0 – 1

Scored outputs

3 outputs

Annual lighting energy kwh

annual_lighting_energy_kwh

Annual lighting energy consumption

Scores if within ±3% of the reference value.

Leni kwh m2 year

leni_kwh_m2_year

Lighting Energy Numeric Indicator

Scores if within ±3% of the reference value.

Reference saving pct

reference_saving_pct

Percentage saving against the reference LENI

Scores if within ±3% of the reference value.

Difficulty

Each template is sampled at three tiers. Harder tiers may hide inputs, forcing the model to infer them from the scenario description.

easy

All inputs given

Office zone with all values visible

medium

All inputs given

Interior lighting zone selected from office or healthcare cases

hard

Some inputs hidden

Healthcare zone with daylight factor embedded in context

Hidden inputs

  • Daylight factordaylight_factor

Prompt replacement text

Use the daylight availability factor implied by {{ archetype.description }}.

Daylight factor restricted to: 0.75

Task bundle

The exact instruction and parameter contract used to generate this task, pinned to the published library source.

/workspace

  • instruction.md
  • leni-calculation_calc.py

Teal lines show Jinja input conditions, not task visibility policy. A line renders only when that input or tool is visible.

1# ABOUTME: Prompt template for Lighting Energy Numeric Indicator tasks.2# ABOUTME: Presents installed power, hours, factors, area, and reference LENI.3 4You are a senior interior lighting engineer checking lighting energy performance.5 6## Given7 8| Parameter | Value | Unit |9|-----------|-------|------|10| Installed lighting power | {{ installed_lighting_power_w }} | W |11| Annual operating hours | {{ annual_operating_hours }} | h/year |12| Control factor | {{ control_factor }} | - |13| Daylight factor | {{ daylight_factor }} | - |14| Zone area | {{ zone_area_m2 }} | m2 |15| Reference LENI | {{ reference_leni_kwh_m2_year }} | kWh/m2/year |16 17## Constraints18 19- Annual lighting energy equals installed power times annual hours times control and daylight factors, divided by 1000.20- LENI equals annual lighting energy divided by zone area.21- Reference saving is the percentage reduction from the reference LENI.22 23## Output Format24 25Include a JSON block with exactly these keys:26 27```json28{29 "annual_lighting_energy_kwh": <numeric_value>,30 "leni_kwh_m2_year": <numeric_value>,31 "reference_saving_pct": <numeric_value>32}33```34 35Write your complete solution to `/workspace/output.md`.36

Scenario archetypes

Each generated task is drawn from one of these realistic scenario bands.

Site contexts ground each scenario in a real locale the model can use to infer hidden values.

Office zone

office_zone

Open office lighting zone

office-floorworkplace-lighting
Parameter ranges
installed_lighting_power_w
1000 – 30000
annual_operating_hours
1800 – 3500
control_factor
0.6 – 0.9
daylight_factor
0.5 – 0.9
zone_area_m2
100 – 3000
reference_leni_kwh_m2_year
12 – 30

Healthcare zone

healthcare_zone

Healthcare interior lighting zone

healthcareclinical-lighting
Parameter ranges
installed_lighting_power_w
5000 – 80000
annual_operating_hours
3000 – 7000
control_factor
0.7 – 1
daylight_factor
0.6 – 1
zone_area_m2
300 – 10000
reference_leni_kwh_m2_year
20 – 60

Example task

healthcare-healthcare-zone-preview — hard difficulty, some inputs hidden.

Healthcare interior lighting zone. healthcare. Required outputs: annual_lighting_energy_kwh, leni_kwh_m2_year, reference_saving_pct

The model sees

Scenario context and visible inputs.

installed_lighting_power_w
5000 to 80000 W
annual_operating_hours
3000 to 7000 h/year
control_factor
0.7 to 1 -
zone_area_m2
300 to 10000 m2
reference_leni_kwh_m2_year
20 to 60 kWh/m2/year

Executable tool: leni-calculation_calc.py

The model must infer

Inputs withheld at this difficulty.

  • Daylight factor

    daylight_factor

Stand-in text in the prompt

Use the daylight availability factor implied by {{ archetype.description }}.

The model must produce

The scored JSON answer schema.

{
  "annual_lighting_energy_kwh": <number>,
  "leni_kwh_m2_year": <number>,
  "reference_saving_pct": <number>
}
  • annual_lighting_energy_kwh · scored within ±3%
  • leni_kwh_m2_year · scored within ±3%
  • reference_saving_pct · scored within ±3%