fix(lifetime-usage): count advance usage only once

[lago-claude-ai-agent]

"Total lifetime usage" counted invoiceable pay-in-advance usage twice: the immediate charge_in_advance invoice lands in invoiced_usage_amount_cents while the same usage is still reported by current_usage_amount_cents for the open period, so one €10 event showed a €20 lifetime total.

LifetimeUsages::CalculateService now subtracts the open-period pay-in-advance invoiced amount from the current usage, keyed off the fee's stored charges_to_datetime — the fee carries the open-period boundaries, the invoice_subscription of an in_advance_charge invoice does not. Since the invoiced counter is cached behind recalculate_invoiced_usage, it is also refreshed whenever that amount is non-zero, so both counters come from one snapshot rather than one of them being stale. After rollover the subtraction stops and the usage stays counted once, on the invoiced side.

This intentionally changes progressive billing: UsageThresholds::CheckService reads these counters, so thresholds now fire at the customer's true cumulative usage instead of roughly double it for pay-in-advance invoiceable charges. The added threshold spec locks that in.

A rake task flags dormant subscriptions for recalculation, as they would otherwise keep serving the doubled total. Terminated subscriptions will not self-heal — CalculateService skips inactive subscriptions.

Perf: calculate_invoiced_usage_amount_cents was an N+1 (a fee query per invoice) and is now a single aggregate, which matters because it can now run on every recalculation.

Out of scope: non-invoiceable pay-in-advance charges have the opposite problem (their usage drops out of the total after rollover), tracked separately.

[lago-claude-ai-agent]

HOLD — the double count is real and the diagnosis looks right, but the fix picks one side of an unnamed fork that moves the meaning of a public API field and changes progressive-billing firing. That needs a human sign-off, not a re-run.

Blocking:

• Two defensible fixes exist and only one is implemented, without the choice being stated: subtract the open-period amount from current_usage_amount_cents (chosen), or leave current usage alone and exclude open-period in_advance_charge fees from invoiced_usage_amount_cents until the period closes. Which counter absorbs open-period pay-in-advance usage is a data-model decision.
• The chosen side changes a public REST field: V1::LifetimeUsageSerializer exposes current_usage_amount_cents, which now reports 0 for a subscription with 10 EUR of open-period usage and no longer agrees with the customer current-usage endpoint for the same period. spec/scenarios/lifetime_usages/pay_in_advance_invoiceable_spec.rb:266 locks that in.
• Progressive billing changes on a premium feature: LifetimeUsages::UsageThresholds::CheckService filters on threshold.amount_cents > invoiced_usage, so thresholds at or below the already-invoiced amount are now skipped. The PR body calls this intentional; it still needs an owner's decision.
• Terminated subscriptions keep the doubled total permanently — CalculateService early-returns for inactive subscriptions and nothing else resets the columns, so the rake task cannot heal them. Documented in a rake comment, not addressed; whether that stale public data is acceptable is a product call.

Non-blocking:

• calculate_invoiced_usage_amount_cents is described as pure perf, but it also drops a latent double count: the old Ruby-level sum over a joins(:invoice_subscriptions) relation counted an invoice's fees once per matching invoice_subscription, which happens when one invoice covers two subscriptions in the same external_id/subscription_at lineage. Worth calling out explicitly and covering with a test.
• The open-period window charges_to_datetime > Time.current is looser than the existing precedent in Fees::ChargeService#already_billed?, which matches charges_from_datetime and charges_to_datetime exactly against the current boundaries.
• [current_usage.amount_cents - pay_in_advance_invoiced_amount_cents, 0].max is required by the model's greater_than_or_equal_to: 0 validation, but it silently masks over-subtraction (fee priced differently from the recomputed usage, e.g. a pricing-unit conversion change) and under-reports the total with no signal.
• The forced-refresh clause is largely redundant with LifetimeUsages::FlagRefreshFromInvoiceService, which already sets recalculate_invoiced_usage when the immediate invoice is finalized; its cost is an aggregate over the subscription's whole invoice history on every event-driven recalculation for the open period.

Verified: CI green (10/10 spec shards, linters, migrations), commit verified and signed off, branch name clean, no secrets, five files, rubocop disables mirror existing rake/spec precedent. The unit specs are genuine regression tests — they assert 1000 where the old code yields 2000.

Note: I could not read the originating ticket or its comments from this environment, so I cannot confirm whether the progressive-billing change and the public-field semantics were sanctioned there. If they were, that is most of this HOLD resolved.

[aquinofb]

The fix looks right to me, and the specs earn their keep: if I revert only calculate_service.rb and keep the new tests, four of the five scenario examples fail, so they really do pin the double count instead of passing either way. I also ran the lifetime usage, progressive billing and advance charges suites together and nothing else moved.

Two things I would like to see changed before merge, and one that deserves its own ticket. They are in the inline comments.

[aquinofb]

This query now runs on every recalculation for every subscription, including plans that have no pay-in-advance charge at all. The condition on properties->>'charges_to_datetime' cannot use an index, so Postgres has to read all of the subscription's pay-in-advance fees and convert the stored date on each one, and for an invoiceable pay-in-advance charge that set grows by one fee per event and never shrinks. The recalculation runs inline on subscription activity, so the busiest subscriptions pay this most often.

A fee whose period ends in the future can only have been created during the current period, so adding fees.created_at >= <current period start> next to the existing condition keeps the same meaning while letting the database skip everything older. Could you run an EXPLAIN for a subscription with a large fee history before we merge, so we know whether that is needed now or later?

[aquinofb]

This is worth a line in the description, because it is not only a faster version of the old query: it also returns a different number in one case. The old code summed a relation joined to invoice_subscriptions, so an invoice covering two subscriptions of the same family, which is what an upgrade produces, had its fees counted twice. On a single fee of 500, main reports 1000 and this branch reports 500.

The new number is the correct one, but it changes lifetime totals and threshold timing for upgraded subscriptions, so I would mention it and add a spec for it. Otherwise someone will fold the subquery back into a join later and quietly bring the double count back.

[aquinofb]

Small ordering detail: because || stops at the first true value, when the flag is already set the subtraction amount is computed after the invoiced sum rather than before it. In that order, a fee invoiced in between is taken off the current usage while still missing from the invoiced side, which is the mismatch this branch exists to prevent and the opposite of what the comment above it promises. Reading the amount into a local variable before the if puts both sides back on the same snapshot.

[aquinofb]

A threshold of 1500 against 1000 of usage only fires when the total is doubled, so this covers the reported case but not the other ordering. The interesting one is a threshold below the invoiced amount: on main the check reports a 500 threshold as passed even though that usage had already been invoiced, and with this change it reports none. Adding that example would lock down the half of the fix that is easier to break by accident.

[aquinofb]

Not for this PR, but I ran into it while reviewing and it is the same family of problem: the invoiced sum counts every charge fee on the matching invoices, not only the ones belonging to this subscription. With one invoice covering two subscriptions of the same customer, with fees of 100 and 900, the first subscription's lifetime usage reports 1000, both before and after this change. Since a customer's subscriptions are billed together on one invoice, any customer with more than one subscription has an inflated lifetime usage today. Better as its own ticket than widening this one.

[lago-claude-ai-agent]

• Snapshot ordering: memoized the open-period amount before recalculating invoiced usage, so both counters use one snapshot.
• Upgrade de-duplication: added a two-subscription family regression where a single 500 fee is counted once; this is a behavioral correction, not only a query optimization.
• Threshold ordering: added the 500-threshold/1000-invoiced scenario asserting no progressive invoice is issued.
• Fee-query plan: this driver has no API container or PostgreSQL, so I could not produce a meaningful large-history EXPLAIN; I left the optional created_at predicate unchanged.
• Cross-subscription fee attribution: kept it out of this PR as requested.

[lago-claude-ai-agent]

HOLD — the new open-period fee lookup adds an unbounded database scan to every activity-driven lifetime-usage recalculation.

• Bound the fee scope to the current charge period (for example, with fees.created_at >= the current period start) so PostgreSQL does not scan and cast the subscription’s entire fee history on this hot path.