Skip to content

refactor: a solver family's cached data is one object, not eighteen members - #187

Closed
BDonnot wants to merge 4 commits into
masterfrom
claude/pr-185-review-ajco9y
Closed

refactor: a solver family's cached data is one object, not eighteen members#187
BDonnot wants to merge 4 commits into
masterfrom
claude/pr-185-review-ajco9y

Conversation

@BDonnot

@BDonnot BDonnot commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

The bus labelling, Ybus / Sbus, the slack, the pv-pq split and the connectivity
snapshot are one picture of the grid taken at one instant, all of it expressed in
ONE bus labelling. They were eighteen separate LSGrid members, and
pre_process_solver took six of them as parameters while reaching for the other
three itself -- so a caller building into its own vectors got six of its outputs
and the grid got three of them.

That is a real defect in the batch path. A TimeSeries / ContingencyAnalysis prep
wrote its pv-pq split and its slack weights into the grid's cache, next to a Ybus
built for a different labelling, and left that grid still claiming the mixture was
reusable (checked against dev_0.13.2: need_reset_solver() stays false across such
a build). The reuse guard then sized one owner's containers against the other's;
sizes agree far more often than labellings do, so a caller reusing its containers
with a "nothing changed" control would have skipped fillpv_pq and stamped this
grid's split onto its own system -- converged, plausible, wrong. Nothing reaches
it today: the batch works on a private copy and always asks for a full rebuild.
Both of those are accidents, and neither is written down as a requirement.

So rather than adding a check that the nine parts agree, make them one object.

src/core/SolverSideCache.hpp
SolverBusLayout -- labelling, slack, pv-pq split, connectivity snapshot,
allow_reuse, algo_needs_rebuild (types that do not
mention the family's scalar)
SolverSideCache -- the above, plus mat and inj (the two that do)
T = cplx_type -> AC (Ybus / Sbus)
T = real_type -> DC (Bbus / Pbus)

LSGrid holds ac_cache_ / dc_cache_. pre_process_solver / pre_process_dc_solver
take one cache reference. There is no way to hand either half of one.

What this deletes, rather than adds:

  • the reuse guard is cache.is_usable(nb_bus), in one place, instead of eight
    comparisons written inline against a mix of two objects' members;
  • "is this my cache?" is &c == &ac_cache_, two overloads picked by the family's
    own type, instead of counting nine pointer comparisons and throwing on a
    partial match;
  • _mark_cache_valid / prevent_*_cache_reuse / init_bus_status lose their
    if(ac) x_ac_ else x_dc_ bodies;
  • BaseBatchSolverSynch's eleven loose members become its own two caches, and the
    read-back of _grid_model.get_ac_pv_solver() -- with the three copies its own
    TODO copies are made here, which is not ideal flagged -- is gone: one call
    fills the whole thing, into vectors the batch owns.

And it is the extension point that was asked for: remote / shared voltage-control
layouts, HVDC droop data, whatever comes next, go in as another field and inherit
the lifetime, invalidation and consistency rules of everything already there.
algo_controler_ (DualAlgoControl) deliberately stays where it is -- it is
threaded through every element container's mutator signature and is already
correctly per family.

Two behaviours change, both in the foreign-build path:

  • a build into a caller's cache never reuses (solver_control, the snapshot
    init_bus_status() compares against and the flags it raises all describe THIS
    grid, and say nothing about someone else's cache);
  • it retires this grid's cache for that family afterwards. The labelling and the
    split still have to be published, because the NR extensions read them back
    through lsgrid_ptr rather than from what the solver was handed -- including
    bus_pq, which fill_voltage_control_solver_data needs and which the old
    write-through supplied by accident. Publishing the matrix too would mean copying
    it, so what is left is a view for the extensions, not a cache: the snapshot is
    cleared and the control raised, so this grid's next own powerflow rebuilds.
  • the grid's own algorithm is no longer reset / reconfigured for a solve it will
    never perform.

ABI: this changes LSGrid's member layout, so a consumer that casts an LSGrid
across a module boundary (gpusim2grid -- see the note that used to sit on
_forced_ref_slack_bus_id) must be rebuilt against these headers. That is already
what docs/solver_plugin.rst requires of plugins ("the same version of
lightsim2grid headers that is installed at runtime ... different BaseAlgo
layout"). Flagged in the changelog.

Cost, callgrind instruction counts (slope between 200 and 1200 powerflows on the
exotic-elements IEEE14 grid, so construction cancels): +200 instr on an AC
powerflow (839 340 -> 839 541, +0.024%) and +214 on a warm-cache DC one
(28 095 -> 28 309, +0.76%, and that is the cheapest powerflow this library can
run). Roughly two thirds of it is not this change: a per-function diff of the
profiles puts +74/pf in compute_results_tsc_rxha_no_amps and +60/pf in an
Eigen::Ref helper, neither of which the diff touches -- inlining decisions that
moved when the translation unit was recompiled. Wall clock shows nothing: the
run-to-run spread on one unchanged binary (2464 -> 2744 ns on the same DC case)
is larger than the gap between the two.

Tests:

  • test_cache_reuse.cpp, "the structural half ... unset_changes": the existing
    [unset_changes] sections all reach unset_changes() through
    allow_cache_reuse(false), ie BOTH families off, so the family that runs is
    stopped by is_usable's first line (if(!allow_reuse)) and those sections pass
    with the rest of is_usable deleted. Leaving ONE family automatic is what
    exercises it; verified to SIGSEGV against a build with is_usable gutted to
    return allow_reuse;.
  • test_cache_reuse.cpp, "a build into someone else's cache ...": a foreign build
    fills the caller's cache completely, publishes what the extensions read, and
    retires the grid's -- so the grid's own next powerflow is unaffected. The
    need_reset_solver() assertion is the one that separates this from dev_0.13.2.
  • test_batch_voltage_control.cpp: the source grid solves identically on both sides
    of a TimeSeries and a ContingencyAnalysis.

Verified: 210 test cases / 5815 assertions pass under Release, under C++14
(LS2G_CXX_STANDARD=14), under ASan+UBSan, and under valgrind (0 errors, no
leaks). The python layer could not be exercised here (no pybind11 / numpy on this
machine); no python-visible API changes.

Signed-off-by: DONNOT Benjamin benjamin.donnot@rte-france.com
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u

BDonnot and others added 4 commits August 29, 2026 15:26
…embers

The bus labelling, Ybus / Sbus, the slack, the pv-pq split and the connectivity
snapshot are one picture of the grid taken at one instant, all of it expressed in
ONE bus labelling. They were eighteen separate LSGrid members, and
`pre_process_solver` took six of them as parameters while reaching for the other
three itself -- so a caller building into its own vectors got six of its outputs
and the grid got three of them.

That is a real defect in the batch path. A TimeSeries / ContingencyAnalysis prep
wrote its pv-pq split and its slack weights into the grid's cache, next to a Ybus
built for a different labelling, and left that grid still claiming the mixture was
reusable (checked against dev_0.13.2: need_reset_solver() stays false across such
a build). The reuse guard then sized one owner's containers against the other's;
sizes agree far more often than labellings do, so a caller reusing its containers
with a "nothing changed" control would have skipped fillpv_pq and stamped this
grid's split onto its own system -- converged, plausible, wrong. Nothing reaches
it today: the batch works on a private copy and always asks for a full rebuild.
Both of those are accidents, and neither is written down as a requirement.

So rather than adding a check that the nine parts agree, make them one object.

  src/core/SolverSideCache.hpp
    SolverBusLayout      -- labelling, slack, pv-pq split, connectivity snapshot,
                            allow_reuse, algo_needs_rebuild (types that do not
                            mention the family's scalar)
    SolverSideCache<T>   -- the above, plus mat and inj (the two that do)
                            T = cplx_type -> AC (Ybus / Sbus)
                            T = real_type -> DC (Bbus / Pbus)

  LSGrid holds ac_cache_ / dc_cache_. pre_process_solver / pre_process_dc_solver
  take one cache reference. There is no way to hand either half of one.

What this deletes, rather than adds:
- the reuse guard is `cache.is_usable(nb_bus)`, in one place, instead of eight
  comparisons written inline against a mix of two objects' members;
- "is this my cache?" is `&c == &ac_cache_`, two overloads picked by the family's
  own type, instead of counting nine pointer comparisons and throwing on a
  partial match;
- `_mark_cache_valid` / `prevent_*_cache_reuse` / `init_bus_status` lose their
  `if(ac) x_ac_ else x_dc_` bodies;
- BaseBatchSolverSynch's eleven loose members become its own two caches, and the
  read-back of _grid_model.get_ac_pv_solver() -- with the three copies its own
  `TODO copies are made here, which is not ideal` flagged -- is gone: one call
  fills the whole thing, into vectors the batch owns.

And it is the extension point that was asked for: remote / shared voltage-control
layouts, HVDC droop data, whatever comes next, go in as another field and inherit
the lifetime, invalidation and consistency rules of everything already there.
`algo_controler_` (DualAlgoControl) deliberately stays where it is -- it is
threaded through every element container's mutator signature and is already
correctly per family.

Two behaviours change, both in the foreign-build path:
- a build into a caller's cache never reuses (`solver_control`, the snapshot
  init_bus_status() compares against and the flags it raises all describe THIS
  grid, and say nothing about someone else's cache);
- it retires this grid's cache for that family afterwards. The labelling and the
  split still have to be published, because the NR extensions read them back
  through `lsgrid_ptr` rather than from what the solver was handed -- including
  bus_pq, which fill_voltage_control_solver_data needs and which the old
  write-through supplied by accident. Publishing the matrix too would mean copying
  it, so what is left is a view for the extensions, not a cache: the snapshot is
  cleared and the control raised, so this grid's next own powerflow rebuilds.
- the grid's own algorithm is no longer reset / reconfigured for a solve it will
  never perform.

ABI: this changes LSGrid's member layout, so a consumer that casts an LSGrid
across a module boundary (gpusim2grid -- see the note that used to sit on
_forced_ref_slack_bus_id) must be rebuilt against these headers. That is already
what docs/solver_plugin.rst requires of plugins ("the same version of
lightsim2grid headers that is installed at runtime ... different BaseAlgo
layout"). Flagged in the changelog.

Cost, callgrind instruction counts (slope between 200 and 1200 powerflows on the
exotic-elements IEEE14 grid, so construction cancels): +200 instr on an AC
powerflow (839 340 -> 839 541, +0.024%) and +214 on a warm-cache DC one
(28 095 -> 28 309, +0.76%, and that is the cheapest powerflow this library can
run). Roughly two thirds of it is not this change: a per-function diff of the
profiles puts +74/pf in compute_results_tsc_rxha_no_amps and +60/pf in an
Eigen::Ref helper, neither of which the diff touches -- inlining decisions that
moved when the translation unit was recompiled. Wall clock shows nothing: the
run-to-run spread on one unchanged binary (2464 -> 2744 ns on the same DC case)
is larger than the gap between the two.

Tests:
- test_cache_reuse.cpp, "the structural half ... unset_changes": the existing
  [unset_changes] sections all reach unset_changes() through
  allow_cache_reuse(false), ie BOTH families off, so the family that runs is
  stopped by is_usable's first line (`if(!allow_reuse)`) and those sections pass
  with the rest of is_usable deleted. Leaving ONE family automatic is what
  exercises it; verified to SIGSEGV against a build with is_usable gutted to
  `return allow_reuse;`.
- test_cache_reuse.cpp, "a build into someone else's cache ...": a foreign build
  fills the caller's cache completely, publishes what the extensions read, and
  retires the grid's -- so the grid's own next powerflow is unaffected. The
  need_reset_solver() assertion is the one that separates this from dev_0.13.2.
- test_batch_voltage_control.cpp: the source grid solves identically on both sides
  of a TimeSeries and a ContingencyAnalysis.

Verified: 210 test cases / 5815 assertions pass under Release, under C++14
(LS2G_CXX_STANDARD=14), under ASan+UBSan, and under valgrind (0 errors, no
leaks). The python layer could not be exercised here (no pybind11 / numpy on this
machine); no python-visible API changes.

Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
init_bus_status() took SubstationContainer::get_bus_status() by value:

    const std::vector<bool> new_status = substations_.get_bus_status();

The accessor already returns `const std::vector<bool> &`, so this copied the
whole vector into a local, to be read twice by _flag_dimension_change and
thrown away. Every mutation of the bus status happens in the disconnect /
reconnect calls above the line, and _flag_dimension_change only reads, so a
reference is safe.

One missing `&`. O(nb_bus) per powerflow that reaches init_bus_status, which is
every powerflow that rebuilds and every one where an element changed bus.

Verified: 210 test cases / 5815 assertions pass under Release.

Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
Same split as compute_pf_with_input_validation / compute_pf: the user-facing
entry point does not take the caller's word for it, the internal one does,
because it has just built the data itself.

  unset_changes()            user-facing, BOTH families, checked
  _mark_cache_valid(...)     internal, one family, no checks

`unset_changes()` still marks both families -- that is its historical contract
and pre-1.0.0 code relies on it -- but each is now verified before the claim is
recorded, and a family that cannot back it is retired rather than marked.

The check that matters is not the one I expected
------------------------------------------------
The obvious check is "does the cache hold a system of the right shape". It is
necessary -- that is what stands between a claim about a family that never
solved and an out-of-bounds read -- but it is nowhere near sufficient, and the
new test caught it: after

    ac_pf(); deactivate_powerline(4); unset_changes(); ac_pf();

the answer was wrong by 0.038 pu. Deactivating that line changes no bus's
connected/disconnected status and no vector's size, so every structural check
passes -- and `tell_none_changed()` then cleared the pending `recompute_ybus`,
so the next powerflow re-solved a Ybus that still contained the line.

A cache cannot tell from its own contents that it is stale. Change an
impedance, a tap, a load target: every size and every bus status is exactly
what it was. What knows is the element containers, which declare every change
through AlgoControl in their own modifiers -- so the load-bearing test is to
READ that back, via the new `AlgoControl::nothing_changed()` (the exact negation
of `tell_all_changed()`). The containers stay the sole authority: nothing here
second-guesses what they declared, adds flags of its own, or recomputes
connectivity behind them.

The powerflow path stops re-checking
-------------------------------------
`_pre_process_solver_impl` now asks only the switch (`!own_cache ||
!cache.may_be_reused()`). It can, because every other way the flags reach it can
only make a cache MORE stale, never falsely fresh: the containers raise them as
the grid is modified, AlgoControl's constructor asks for a full rebuild,
set_state / the copy ctor / a divergence reset, and python cannot clear them
(`get_*_algo_controler()` is bound read-only -- binding_misc.cpp exposes the
has_* / need_* getters and no tell_*). The two entry points that CAN claim
"nothing changed" without having built anything now verify it themselves:
`unset_changes()`, and `check_solution()`, whose weaker
`id_me_to_solver.size() > 0` guard is replaced by the same consistency check.

A debug-only assertion keeps that reasoning honest: free under -DNDEBUG (what
the wheels ship), and it fires in the C++ suite -- which CI runs under ASan,
UBSan and valgrind -- the day a third claimant appears. Verified non-vacuous:
making `unset_changes()` mark unconditionally aborts the Debug build.

`SolverSideCache`'s predicates are now two separate questions, which is what
made the above possible to reason about at all:
  is_consistent(nb_bus)   is the data there and self-consistent (sizes only)
  may_be_reused()         is reuse allowed at all
Neither asks "has the grid changed" -- that is not a question a cache can answer
about itself, and AlgoControl already answers it.

Tests: test_cache_reuse.cpp's [unset_changes] case rewritten. It used to pin the
dangerous intermediate state -- that a never-built family got marked "in sync",
with the powerflow path catching it later. It now pins the opposite: the claim
is refused at the point it is made. Sections cover a family that never solved,
each family solved alone, a pending topology change (the 0.038 pu case), a
genuinely valid cache claimed repeatedly, and the three sequences that used to
segfault. Verified the flag test bites: removing it reproduces the wrong answer.

Verified: 210 test cases / 5830 assertions pass under Release, C++14, Debug
(assertions live), ASan+UBSan, and valgrind (0 errors, no leaks). The python
layer could not be exercised here (no pybind11 / numpy on this machine).

Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UtvkqXMvgCig75gruZCs7u
Review feedback on #185: now that a family's solver-side data is one
object, reset() should say so.

The body was the pre-1.0.1 one with the member names substituted -- ten
assignments, split across three blocks, that between them clear exactly
what SolverSideCache::clear() clears. `if(reset_ac) ac_cache_.clear();`
is the same thing said once.

Two things that were NOT the same, both consequences of the old split
rather than choices:

- The per-family flags cleared only half a cache. `reset_ac` gated the
  labelling and the matrix; the injections, the pv-pq split and the
  slack were cleared for BOTH families whatever the flags said, leaving
  a non-reset family holding a labelling and a matrix with no slack and
  no split. That half-state fails is_consistent(), so nothing could
  reuse it -- it was unreachable rather than harmless. The flag now
  means what its name says. Every caller in the tree passes
  (true, true, true), so nothing changes for them.

- `algo_needs_rebuild` was cleared unconditionally under the comment
  "the algorithms themselves are reset below", which is true only when
  reset_solver is set. It moves inside that branch, next to the resets
  that justify it: with reset_solver false the algorithms keep their
  internals and this flag is the only thing left that would tell them to
  rebuild from the cache.

`tell_solver_need_reset()` -> `prevent_cache_reuse()`, its own preferred
name; same function.

210 test cases / 5830 assertions pass, Release and Debug/C++14 (the
latter with the powerflow path's debug assertions live).

Signed-off-by: DONNOT Benjamin <benjamin.donnot@rte-france.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019y5RMtrKWt9PQEXvjqhmpn
@BDonnot
BDonnot force-pushed the claude/pr-185-review-ajco9y branch from 1fd0c7a to 55bfdad Compare August 29, 2026 18:10
@BDonnot BDonnot closed this Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant