Add support for InfraStore.jl - #212
Draft
daniel-thom wants to merge 27 commits into
Draft
Conversation
daniel-thom
force-pushed
the
feat/rust-time-series-store
branch
from
July 30, 2026 00:42
a54e0ec to
7ddf427
Compare
…supply
One demand curve on a PSY.GroupReserve is cleared by the summed awards of its
contributing services: a dense ServiceRequirementVariable per group, a clearing
constraint sum(member awards) >= demand variable (its dual is the group price),
and the group curve priced through the delta-PWL path. Static and time-series
group curves are both supported via the existing service-side TS machinery.
- Group demand predicates mirror the service formulations: GroupRangeReserve is
driven by the scalar requirement, GroupStepwiseCostReserve by the demand curve;
degenerate groups skip as supply aggregates with a warning.
- Group deferral generalized to a vector so up and down groups coexist.
- RESERVE_PRODUCT_TYPES (definitions.jl) consolidates the open
Union{PSY.AbstractReserve, PSY.GroupReserve} signature bound used across the
reserve traits, PWL parameter chain, and objective plumbing.
- Formulation-pairing guards: a GroupReserve accepts only group formulations and
vice versa, failing with ArgumentError at ServiceModel declaration.
- Tests cover build/solve, aggregation binding, merit order, no-group baseline,
the degenerate skip, TS group curves, and the pairing guards.
PSY moved GroupReserve into the reserve tree, so the RESERVE_PRODUCT_TYPES
alias and the per-type methods that existed only because groups sat outside
it are gone: every former Union bound is plain PSY.AbstractReserve, the
group get_initial_conditions_service_model and the CostExpressions container
sibling fold into the AbstractReserve methods, and uses_compact_power opens
to the abstract type. Formulation-pair bounds
(Union{StepwiseCostReserve, GroupStepwiseCostReserve}) and the group demand
predicates stay - they encode formulation semantics, not typing.
The pairing guards merge with the #235 hardening set: valid direction-applied
pairs for both group formulations, the generic-defaults disambiguator, the
inverse guard over both formulations, and a direction-required error for
bare GroupReserve declarations.
A controllable load routes reserves inversely to generators: up reserve is committed shed (P - r_up >= 0), down reserve is committed extra consumption (P + r_down <= forecast). Dispatch limits move to the range expressions only when a reserve service is attached, and a costless load selling reserves fails loudly since nothing pins its consumption.
…-out End-to-end market test: an elastic OnlineReserve (StepwiseCostReserve) and a GroupStepwiseCostReserve group co-clear against per-resource offers from thermal, storage, and load participants. Registers GroupStepwiseCostReserve in the formulation library, refreshes the stale group-reserve warnings there, and renames the remaining market-specific reserve identifiers in hydro to generic ones.
Non-spinning is upward-only, so OfflineReserve routes like an up reserve everywhere a device supplies it. New UP_RESERVE union in reserve_traits.jl; storage reserve-balance multipliers, coverage branches (two of which silently skipped OfflineReserve, one asserted), get_fraction, and the TotalReserveOffering fold widened; load routing and folding accept it as committed shed. The _modify_device_model! no-op is scoped to NonSpinningReserve, whose awards ride ReservePowerConstraint instead of the device range expressions.
PowerSystems psy6 (post schema-matching merge) depends on the unregistered PowerCoreOpenAPIModels / PowerOperationsOpenAPIModels. Pkg ignores [sources] of non-root projects, so each environment that resolves PSY - root, test, docs - must pin them itself; CI failed with 'PowerOperationsOpenAPIModels has no known versions' on all jobs. Pins mirror PSY's own (monorepo main, subdirs) and are temporary until the packages are registered.
…supply
One demand curve on a PSY.GroupReserve is cleared by the summed awards of its
contributing services: a dense ServiceRequirementVariable per group, a clearing
constraint sum(member awards) >= demand variable (its dual is the group price),
and the group curve priced through the delta-PWL path. Static and time-series
group curves are both supported via the existing service-side TS machinery.
- Group demand predicates mirror the service formulations: GroupRangeReserve is
driven by the scalar requirement, GroupStepwiseCostReserve by the demand curve;
degenerate groups skip as supply aggregates with a warning.
- Group deferral generalized to a vector so up and down groups coexist.
- RESERVE_PRODUCT_TYPES (definitions.jl) consolidates the open
Union{PSY.AbstractReserve, PSY.GroupReserve} signature bound used across the
reserve traits, PWL parameter chain, and objective plumbing.
- Formulation-pairing guards: a GroupReserve accepts only group formulations and
vice versa, failing with ArgumentError at ServiceModel declaration.
- Tests cover build/solve, aggregation binding, merit order, no-group baseline,
the degenerate skip, TS group curves, and the pairing guards.
PSY moved GroupReserve into the reserve tree, so the RESERVE_PRODUCT_TYPES
alias and the per-type methods that existed only because groups sat outside
it are gone: every former Union bound is plain PSY.AbstractReserve, the
group get_initial_conditions_service_model and the CostExpressions container
sibling fold into the AbstractReserve methods, and uses_compact_power opens
to the abstract type. Formulation-pair bounds
(Union{StepwiseCostReserve, GroupStepwiseCostReserve}) and the group demand
predicates stay - they encode formulation semantics, not typing.
The pairing guards merge with the #235 hardening set: valid direction-applied
pairs for both group formulations, the generic-defaults disambiguator, the
inverse guard over both formulations, and a direction-required error for
bare GroupReserve declarations.
A controllable load routes reserves inversely to generators: up reserve is committed shed (P - r_up >= 0), down reserve is committed extra consumption (P + r_down <= forecast). Dispatch limits move to the range expressions only when a reserve service is attached, and a costless load selling reserves fails loudly since nothing pins its consumption.
…-out End-to-end market test: an elastic OnlineReserve (StepwiseCostReserve) and a GroupStepwiseCostReserve group co-clear against per-resource offers from thermal, storage, and load participants. Registers GroupStepwiseCostReserve in the formulation library, refreshes the stale group-reserve warnings there, and renames the remaining market-specific reserve identifiers in hydro to generic ones.
Non-spinning is upward-only, so OfflineReserve routes like an up reserve everywhere a device supplies it. New UP_RESERVE union in reserve_traits.jl; storage reserve-balance multipliers, coverage branches (two of which silently skipped OfflineReserve, one asserted), get_fraction, and the TotalReserveOffering fold widened; load routing and folding accept it as committed shed. The _modify_device_model! no-op is scoped to NonSpinningReserve, whose awards ride ReservePowerConstraint instead of the device range expressions.
PowerSystems psy6 (post schema-matching merge) depends on the unregistered PowerCoreOpenAPIModels / PowerOperationsOpenAPIModels. Pkg ignores [sources] of non-root projects, so each environment that resolves PSY - root, test, docs - must pin them itself; CI failed with 'PowerOperationsOpenAPIModels has no known versions' on all jobs. Pins mirror PSY's own (monorepo main, subdirs) and are temporary until the packages are registered.
…na/PowerOperationsModels.jl into rh/new_load_and_group_services
…serves The assignment bounds applied the discharge-side convention (up raises power, down lowers it) to BOTH sides, but reserves swap roles on the charge side: a downward reserve INCREASES charging and an upward reserve DECREASES it - the same convention the deployment expressions already implement. As written, charge-side down-reserve room was capped at p_in (zero for an idle or discharging storage) instead of charge_max - p_in, and charge-side up room was over-granted as charge_max - p_in instead of p_in. Total up room for an idle storage exceeded the HSL - p_net capability; total down room collapsed to p_out. Map the assignment bounds in side-increasing/decreasing terms instead (discharge: Up/Down; charge: Down/Up).
…tchWithReserves New attribute, default true (existing behavior unchanged). Setting false decouples ancillary services from the energy schedule, matching day-ahead market clearing where AS awards are bounded by offer quantity and capability only: - no ReserveCoverageConstraint / ReserveCoverageConstraintEndOfPeriod (SOC feasibility of awards is a real-time concern, not a clearing rule) - reserve deployment bounds use the no-reservation builders, so the AS band never depends on the reservation binary; the binary still governs energy charge/discharge exclusivity - the SOC evolution carries no expected reserve-deployment energy - complete_coverage is suppressed with a warning instead of silently combined Regression test builds a decoupled reserve-carrying storage and asserts the coverage constraints are absent, the reservation binary exists, and no deployment power-limit row references it.
Without the self-pin a clean clone resolves POM from the registry and tests run against a released version instead of the checkout.
A load that sells reserves with no price on its consumption has nothing pinning that consumption, so the build rejects it. The check only recognized a zero LoadCost, but a ControllableLoad can carry a MarketBidCost just as well, and MarketBidCost defaults to a zero offer curve -- so the case the check exists to catch walked straight past it. Dispatch the emptiness test on the cost type instead of branching on isa, which also gives the market-bid variants somewhere to live: both read the decremental side, the one a load offers on, and reuse is_nontrivial_offer to tell a real curve from PSY's zero placeholder.
The comment said the test was disabled pending upstream work, but nothing disabled it and the sentence broke off mid-thought. The upstream cause is fixed: importing a zero-capacity reservoir no longer divides 0 by 0 into a NaN initial_level, which is what made solve! fail while writing the system out (PowerSystems 7b27254f8). The assertion passes, so the note is just misleading.
…ies-store # Conflicts: # Project.toml # src/common_models/add_parameters.jl # src/network_models/network_reductions.jl # test/Project.toml
Replace `PSY.to_json(sys, file)` in both solve paths with `PSY.to_file(sys, dir; unit_system = :device_base)`. A serialized System is now a directory (document plus time-series sidecar), so the output member is named by `IOM.make_system_dirname` rather than a `.json` filename. Device base, not the `:original` default: that one reproduces the document a System was read from and needs the round-trip ledger, which a model's system carries only when it was built from a document. `IS.get_uuid` no longer exists — components and supplemental attributes are identified by an integer IS id, and the System's UUID is the only one left. So outage identity moves from `Base.UUID` to `Int` throughout template validation, network-model instantiation, reduction exceptions, and security-constrained branches; `PSY.get_supplemental_attribute` already took an `Int`, so those calls were passing the wrong type. The system-UUID seam `IOM.get_system_uuid` now resolves to `PSY.get_system_uuid`. Drop `scaling_factor_multiplier` from the test fixtures: it was removed from IS, since time series store actual per-device quantities and nothing rescales them on retrieval. This does not build until the upstream branches it pins are pushed: PSY and IS on feat/infrastore-integration (PSY.get_system_uuid), IOM on feat/rust-time-series-store (make_system_dirname), and PNM/PSB on psy6.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thanks for opening a PR to PowerOperationsModels.jl, please take note of the following when making a PR:
Check the contributor guidelines