EIA profile-based powerplant age imputation by ddahawkins-TUDelft · Pull Request #31 · modelblocks-org/module_powerplants

ddahawkins-TUDelft · 2026-07-08T13:23:32Z

Fixes #

Summary of changes in this pull request

Adds a deterministic alternative to group-average powerplant age imputation.

The new method:

derives annual commissioning weights from positive changes in reference capacity stock;
preserves observed commissioning years;
allocates missing-capacity plants against the remaining target profile;
respects technology lifetime constraints;
writes row-level and annual profile diagnostics as Parquet;
produces diagnostic plots comparing observed, imputed, and target profiles.

Validation

Tested on:

MEX wind, solar, and fossil datasets;
all European categories;
multi-country diagnostic profile outputs.

Notes

Powerplant categories are harmonised with the reference statistics categories for:

bioenergy → biomass and waste
fossil → fossil fuels

Reviewer checklist

There are no pip dependencies in the module's environment files (workflow/envs/).
All rules use pathvars (e.g., <results>) in their inputs and outputs.
The integration test-suite is successful, including:
- pre-commit.ci tests pass.
- tests pass for all relevant OS configurations (linux, osx, windows).
Module documentation is up-to-date, including:
- INTERFACE.yaml mentions all relevant pathvars and wildcards.
- README.md describes how to use the module and has the necessary citations.

…ine based on dated capacities.

for more information, see https://pre-commit.ci

ddahawkins-TUDelft · 2026-07-08T13:33:35Z

Adding comment. The new method is only applied to assets with the 'operating' status. It is not yet applied to historical (retired) nor future statuses (pre-construction, construction, announcements). Once you are happy with the methodology as a concept, I can extend it as these other categories will require slightly different target profiles e.g. a linear forecast for the future ones.

for more information, see https://pre-commit.ci

…te Events' parquet and pdfs.

for more information, see https://pre-commit.ci

ddahawkins-TUDelft · 2026-07-21T21:37:45Z

@irm-codebase I have just pushed a final commit that I think addresses your comments above (see resolved) and also adds the 'future assets' capability that you can test. I have endeavored to remove the 'bloat' code but let me know if you catch something.

Please note that in this new version, I import your _utils into _schema as I use _utils to host a single-version-of-truth dictionary of the date imputation sources (see the object: DATE_SOURCE_METADATA in _utils). I use this in _schema to validate the results and in the impute_ages code to assign those labels.

irm-codebase

Unfortunately the code is not running at the moment.
I can only point some structural stuff for the time being.

irm-codebase · 2026-07-25T12:27:32Z

Not used. I assume it can be deleted?

irm-codebase · 2026-07-25T14:24:27Z

+def date_source_labels() -> dict[str, str]:
+    """Return date-source display labels."""
+    return {
+        source_type: metadata["label"]
+        for source_type, metadata in DATE_SOURCE_METADATA.items()
+    }


Suggestion: this kind of label boilerplate code can be avoided with the help of the inflection library, which can be installed like so:

pixi add --feature module inflection # installs the lib for the module pixi run export-snakemake-env module # export the environment so rules can use it

Then just run:

from inflection import humanize print(humanize("imputed_capacity_profile") # Imputed capacity profile

This will make the the name easy to read while matching the dataset naming, which helps avoid confusion. It also forces the dev to come up with concise but clear naming too 😉

I understand what you are asking, but humanize() doesnt add all the context that might helper a user unless i make all the variable names very very long. So I would prefer to keep my DATE_SOURCE_METADATA as-is .... it provides a "single source of truth' generally for this rule so it would have to exist anyway.

irm-codebase · 2026-07-25T14:59:55Z

+DATE_SOURCE_METADATA = {
+    "observed": {


This could be removed with some smart helpers. See below.

for more information, see https://pre-commit.ci

ddahawkins-TUDelft · 2026-07-30T10:07:43Z

Okay, i have uploaded an updated PR that passes my local tests and implements most of your feedback. Can you run your tests for it please?

irm-codebase

I've now run this and I identified several bugs. I'll detail them below.

irm-codebase · 2026-07-31T09:07:07Z

+args = ["env", "platform", "outdir"]
+depends-on = [
+  { task = "_export-conda-explicit-spec", args = ["{{ env }}", "{{ platform }}", "{{ outdir }}"] },
+]
+cmd = "mv '{{ outdir }}/{{ env }}_{{ platform }}_conda_spec.txt' '{{ outdir }}/{{ env }}.{{ platform }}.pin.txt'"
+
+[tasks._export-conda-explicit-spec]
+description = "Export one Pixi environment/platform as an explicit conda spec file"
+args = ["env", "platform", "outdir"]
+cmd = "pixi workspace export conda-explicit-spec --environment '{{ env }}' --platform '{{ platform }}' '{{ outdir }}'"


Why was this changed? Did it not work on windows?
These commands are standard in all modules, so this would imply a higher issue in our templating.

Wait... what? This new code is the code from the template.
Maybe I did not catch that pixi.toml had a regression in the previous review?

The cause is that this branch sits before the latest template update. I should've seen that.
After this there should be no required changes to pixi.toml

irm-codebase · 2026-07-31T10:37:12Z

+def _build_reference_retirement_profile(
+    reference_capacity_df: pd.DataFrame,
+    country_id: str,
+    categories: list[str],
+    years: pd.Index,
+) -> pd.DataFrame:
+    """Build an annual retirement profile from capacity-stock reductions."""


Seems like retirements are assigned oddly in some cases. For the MEX testcase

powerplant_id: GEM_L100000832511-G100000838075

name: is Huerto Solar Durango solar farm---

This plant has no start or end date in the GEM dataset, meaning it will be imputed with the capacity profile (red bit at the start). However:

It is assigned extremely early

The start date is mislabled as 'observed', when it was imputed.

Ok, the right-hand side of this is fixed now.
We still have a left-hand side issue, as the powerplant is being assigned an unlikely date.

I do not think Mexico was that much of a pioneer when it comes to PV ;)
The solution would be to re-run the imputation of start-dates for these plants so they fit within the commissioning profile.

This is a nice-to-have as I highly doubt most people will use this module for historical profiling.
We should still document this as an issue for a later date, though.

irm-codebase

Looks great @ddahawkins-TUDelft !

One thing missing is the README update, and better handling of the start year of retired facilities that have neither a start or end date.
I'll submit a PR for that soon-ish.

ddahawkins-TUDelft and others added 5 commits July 7, 2026 16:56

Version which implements the greedy, largest-smallest allocation rout…

4a9f7b7

…ine based on dated capacities.

Updated EIA target capacity allocation routine

1f79a5f

Working version with cleaned code

5700a53

Added some comments to explain

8b8c7df

[pre-commit.ci] auto fixes from pre-commit.com hooks

5f914f5

for more information, see https://pre-commit.ci

irm-codebase self-requested a review July 8, 2026 13:25

ddahawkins-TUDelft and others added 2 commits July 9, 2026 11:46

Capacity Profile Method Now Addresses Retired Assets

6dfe258

[pre-commit.ci] auto fixes from pre-commit.com hooks

ed6753b

for more information, see https://pre-commit.ci

irm-codebase requested changes Jul 9, 2026

View reviewed changes

ddahawkins-TUDelft and others added 8 commits July 9, 2026 13:33

Updated figures to include retirement profiles

3a33748

Working version

557632a

Working version with Planned Statues and addresses original PR concerns

998b702

[pre-commit.ci] auto fixes from pre-commit.com hooks

9875124

for more information, see https://pre-commit.ci

Added schema validation for start year and end year imputation sources

33ea138

[pre-commit.ci] auto fixes from pre-commit.com hooks

9796752

for more information, see https://pre-commit.ci

Commit removes excessive diagnostics and leaves only the 'Capacity Da…

4bef4fd

…te Events' parquet and pdfs.

[pre-commit.ci] auto fixes from pre-commit.com hooks

46ff3ae

for more information, see https://pre-commit.ci

irm-codebase self-requested a review July 25, 2026 12:22

irm-codebase requested changes Jul 25, 2026

View reviewed changes

ddahawkins-TUDelft and others added 5 commits July 30, 2026 09:52

Updated capacity stock calls

0733140

Update Pixi Snakemake environment export task

1d179be

Addressed PR feedback

8c1a9a1

Adjusted plot colours and fixed dodgy fucntion call bug

c06dbad

[pre-commit.ci] auto fixes from pre-commit.com hooks

4ce0e98

for more information, see https://pre-commit.ci

irm-codebase added 2 commits July 31, 2026 11:13

Small non-code fixes from review 2

8afbc1e

Merge branch 'main' into feature/deterministic-age-imputation

1555409

irm-codebase requested changes Jul 31, 2026

View reviewed changes

ddahawkins-TUDelft and others added 3 commits July 31, 2026 13:50

Fix time imputation edge cases

6994c30

Remove unused time imputation colour helper

f144acb

revert arrangement of functions (for review)

8736400

irm-codebase self-requested a review August 3, 2026 09:20

irm-codebase approved these changes Aug 3, 2026

View reviewed changes

Small fixes to schema checks

e38d53f

irm-codebase merged commit 25b2e3b into modelblocks-org:main Aug 3, 2026
4 checks passed

Uh oh!

Conversation

ddahawkins-TUDelft commented Jul 8, 2026

Summary of changes in this pull request

Validation

Notes

Reviewer checklist

Uh oh!

ddahawkins-TUDelft commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ddahawkins-TUDelft commented Jul 21, 2026

Uh oh!

irm-codebase left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ddahawkins-TUDelft commented Jul 30, 2026

Uh oh!

irm-codebase left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

irm-codebase left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants