feat(patch): meta-population model over the three affected provinces - #412
feat(patch): meta-population model over the three affected provinces#412seabbs-bot wants to merge 52 commits into
Conversation
|
Review pass done (4 inline comments above). CI status: One further observation from watching the fit matrix fan out: So the patch model has unit tests ( This was opened by a bot. Please ping @seabbs for any questions. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #412 +/- ##
==========================================
- Coverage 93.93% 93.64% -0.30%
==========================================
Files 15 16 +1
Lines 3464 3949 +485
==========================================
+ Hits 3254 3698 +444
- Misses 210 251 +41 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
📖 Documentation preview is ready! View the docs for this PR at: http://epiforecasts.io/BVDOutbreakSize/previews/PR412/ This preview will be updated automatically when you push new commits. |
|
This needs to be the new headline model ie joint and be wired into the analysis and sensitivity ie for one week ahead forecasts. I don't see evidence it works here |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
There are some conflicts. The much larger outbreak size is worrying and I am not sure plausible. You don't show estimates or forecats for any of the spatial patches or Rt by patch |
A meta-population / patch model formulation for the 2026 DRC BVD outbreak that splits the single-patch national model into three spatial patches (Ituri, Nord-Kivu, Sud-Kivu). Each patch has a province-specific reproduction number (hierarchical modifier on the national Rt), within-patch renewal transmission, and a between-patch importation kernel. National data streams are fitted to the sum of patch trajectories; per-province spatial-table data (confirmed cases, lab analysed, isolation) fit individual patches. See notes/patch-model.md for the full formulation. Relates-to: patch-model exploration
- patch_rt_model: hierarchical Rt (national walk + per-patch modifiers) - patch_infection_model: multi-patch renewal with importation - patch_infections() and importation_from_kernel() in renewal.jl - Verified: module compiles, prior predictive sampling runs Relates-to: patch-model exploration
- _patch_latent: wraps patch_infection_model and sums patch trajectories - bvd_patch_joint: full joint model using patch latent process with national-level observation submodels - All models compile; patch_infection_model prior predictive verified Relates-to: patch-model exploration #5
Use the national-level cases_state.bvd_reports_daily and cases_state.bg_daily for the treatment flow model, matching the signature of treatment_flow_model. Fix field name from deaths_state.onsets_to_deaths (does not exist) to the correct cases_state.bvd_reports_daily + cases_state.bg_daily pair. Remove duplicate := deterministics for parameters already traced inside the patch submodel (importation_epsilon, sigma_region, C_T_total).
- Expose C_T_patch_1/2/3, R_T_patch_1/2/3, infections_T_patch_1/2/3 as := deterministics in bvd_patch_joint - Add patch_summary_table() to summaries.jl — per-patch posterior summary of C_T, R_T, daily infections, and δ modifier - Export patch_summary_table from BVDOutbreakSize.jl
…onal Rt Replace the constant-modifier patch_rt_model with patch_rt_mvwalk_model: - Each patch gets its own log-Rt trajectory via an MVN random walk at weekly knots - Innovations are correlated across patches via LKJ(2) sampled correlation matrix - Each patch has its own step SD sigma_rw_patch and initial R0 variation - Allows Ituri (high Rt, sustained transmission) and Sud-Kivu (near-zero local Rt) to diverge arbitrarily Add implied_national_Rt() helper to renewal.jl: - Derives the national Rt from summed patch infections via the renewal equation - No separate national Rt parameter — it emerges from the aggregated dynamics Update patch_infection_model to use the new MV walk by default: - Removes rt_state.delta_patch and rt_state.sigma_region dependencies - Adds implied_Rt_national to the return tuple - Exposes sigma_rw_patch and Omega from the MV walk - Fix docstring to document new return values Add LinearAlgebra as a project dependency for cholesky factorization.
Replace delta_patch references with sigma_rw patch step SDs from the multivariate normal random walk.
- Fix intervention ramp in patch_rt_mvwalk_model: add intervention_effect parameter and apply ramp properly (was multiplied by 0.0) - Add @inbounds to MV walk core loops (Sigma_half, knot innovations, daily interpolation) - Add effect_prior keyword arg to patch_rt_mvwalk_model - Fix background_re code path: guard against empty histories with bg_lead pattern matching bvd_joint - Update design doc to document MV walk model as Option C
- Add confirmed_cases_patch_model to observations.jl: shares receipt delay and test sensitivity with the national confirmed stream, fits against per-province data from spatial tables (Tableau 1) - Wire into bvd_patch_joint: per-province fitting loop routes each patch's onsets through the shared lab pipeline - Add province_confirmed_history to load_observations() return (empty Dict by default, ready for data population) - Export s_test and spec from confirmed_cases_model so per-province models can access them - Export confirmed_cases_patch_model from BVDOutbreakSize.jl
- Add province_confirmed_history TOML block with per-province spatial
table data (Ituri, Nord-Kivu, Sud-Kivu confirmed cases from Tableau 1)
- Add province_history() parser in data.jl that reads per-province arrays
from TOML blocks into Dict{String, NamedTuple}
- Wire province_history() output into load_observations return tuple
- Fix MustNotOverwriteError: use = instead of := for
expected_patch_confirmed (loop runs multiple times)
The per-province confirmed counts are an exact partition of the national confirmed counts (verified: Ituri + Nord-Kivu + Sud-Kivu equals the national total at all 17 shared vintages). Fitting them with their own count likelihood alongside the national confirmed stream put the same observations into the joint density twice, double-weighting the confirmed stream against every other stream. Replace confirmed_cases_patch_model with province_composition_model, which factorises P(y) = P(N) x P(shares | N): the national stream keeps the total term, and the composition term scores only the spatial split, by stick-breaking over patches with an overdispersed Binomial. The vintage totals are conditioned on, never scored. Rt across space: the provincial shares are flat over the whole window (Ituri 91.4% -> 91.1%; Ituri grew 1.82x, Nord-Kivu 1.94x), so the data carry no signal for a time-varying divergence between provinces. Drop the multivariate-normal random walk on joint per-patch log-Rt, which fitted ~30 parameters to that absent signal, discarded the national Rt walk the headline model depends on, and had an inverted intervention sign (half normal on lower=0, raising Rt post-intervention, where rt_walk_model uses upper=0). patch_rt_model now layers a constant per-patch modifier on the unchanged national walk, reference-coded on the primary patch (delta_1 = 0) so the walk level and the mean of delta are not confounded. Importation: epsilon is now sampled only when a non-zero kernel is supplied. There is no mobility data, and under flat shares importation is confounded with the secondary-patch seeds, so the default kernel is zero and no epsilon enters the parameter space. Previously epsilon was sampled against an all-zero kernel, giving a parameter the likelihood never touched. Also: surface C_T, R_T, r, r0, T, CFR, R0 and doubling_time under the same names as bvd_joint so a patch chain drops into summary_table and the existing reporting unchanged; rewrite patch_summary_table, which reported the midpoint of the 20-80 percentile range as a "median"; export the patch API, which was not actually exported. Add test/test_patch_model.jl (198 assertions), covering the renewal equivalence, the importation kernel, the implied-national-Rt identity, the composition's invariance to the modelled level, the epsilon gating, and the headline quantities on a real chain. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
Three changes, all driven by what the completed fit actually showed. 1. Secondary-patch seed is now a FRACTION of the primary seed. The absolute prior N+(0.01, 0.01) put the secondary patches ~4 orders of magnitude below Ituri's 2^m ~ 164 -- a seed ratio of ~13,700:1. With only two routes to infections (own seed, own Rt), the log-Rt deviation was forced to absorb the entire level difference. The completed fit confirms it: seed_scale pinned at its prior (0.019, 90% [0.008, 0.034]), and log_rt_contrast[Nord-Kivu] = +0.22, i.e. the model claiming Nord-Kivu transmits 25% MORE than the Ituri epicentre, with the fitted provincial share drifting 5.5% -> 11.7% against a flat observed ~9% and composition coverage of just 7/19 vintages (37%, nominal 90%). seed_fraction ~ LogNormal(log(0.05), 1) puts the seed on the scale the data speak to: the seed explains the LEVEL of the provincial split and the deviation is identified by its TIME TREND. Pinned by a test that the prior brackets the observed split at zero Rt difference. 2. Between-province importation is ON by default. The outbreak spread from Ituri into Nord-Kivu and Sud-Kivu; a meta-population model that cannot move infections between patches is not describing what happened. Default kernel is a gravity kernel weighted by destination population (province_importation_kernel), zero diagonal, with the intensity carried by the sampled epsilon. Passing an all-zero kernel still switches the coupling off and drops epsilon from the parameter space. There is no mobility data, so the kernel is a structural assumption and epsilon is weakly identified against the seeds -- documented, not hidden. 3. Province ascertainment is sampled and partially pooled. The composition weights each patch by asc_p * lambda_p (relative case-finding times modelled incidence), and the data identify only the PRODUCT: a province with fewer infections but better case-finding looks exactly like one with more infections and worse case-finding. Fixing asc_p equal across provinces hides that, and is known to be wrong here -- Ituri ran 31.8% test positivity against Nord-Kivu's 5.5%, so they are testing very differently-selected pools, and forcing equal ascertainment pushes the whole difference into the provincial Rt. asc_p is now sampled, partially pooled toward equality on the log scale and sum-to-zero (only relative ascertainment enters a composition). tau_asc -> 0 recovers the equal-ascertainment model. The pooling prior is what identifies asc_p, so the per-patch results are correspondingly wider -- that is the honest width. Surfaced as province_ascertainment; read it together with log_rt_contrast, since neither is interpretable alone. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
bvd_patch_joint had unit tests but was never fitted in CI: no entry in the fits registry, nothing under scripts/ constructing it, no release artefact. That mattered. Both defects the patch model shipped with were invisible to the unit tests and only surfaced on a real fit: - an AD-breaking Dict lookup inside the model body (every NUTS gradient aborted under Mooncake; prior sampling and log-density evaluation both worked, so nothing caught it) - a secondary-patch seed prior ~4 orders of magnitude too small, which forced the provincial log-Rt deviation to absorb the entire case-split level and made the reported provincial Rt gap an artefact of the prior Registering a `patch` fit makes the posterior-predictive check a standing gate rather than a manual step. The per-province data is reshaped once at registry level, outside the model body, for the AD reason above. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
The per-province CASE split cannot separate incidence from case-finding. A province's confirmed count is the product of its ascertainment and its incidence, and only that product is observed, so a province with fewer infections but better case-finding is observationally identical to one with more infections and worse case-finding. Partial pooling on ascertainment (previous commit) makes that uncertainty honest, but it does not resolve it, and nothing in the case or laboratory data can: the analysed denominator cancels out of the normalised composition shares. Deaths break the tie, and Tableau 1 has been carrying them all along -- only the cases column was ever scanned. Deaths are far harder to miss than cases, and the case-fatality ratio and the death-confirmation probability belong to the virus and to a national laboratory pipeline, not to a province. They are therefore common factors and cancel out of the normalised DEATH shares, leaving a composition that weights each patch by its delay-convolved incidence alone -- free of case ascertainment. The deaths pin lambda_p; the case composition then identifies asc_p as the residual. The identifying assumption is that death ascertainment is near-uniform across provinces, which is far weaker and more defensible than assuming case ascertainment is. The signal is large and sustained: Nord-Kivu holds a steady 8-9% of confirmed cases but 14-19% of confirmed deaths at every one of the 20 vintages (confirmed CFR 54-59% against Ituri's 20-33%). Crucially, some of that gap is NOT ascertainment. A fast-growing epidemic biases the observed CFR down, because recent cases have not yet died, and Ituri grows faster than Nord-Kivu -- a naive CFR comparison would read that right-censoring as a difference in case-finding. The death composition convolves each province's OWN incidence curve through the shared onset-to- death and report-to-receipt delays, so the censoring is accounted for and only the residual is attributed to ascertainment. - scripts/scan_province_tableau1.jl (+ `task province-tableau1`) scans per-province cases AND deaths, gated on both summing exactly to the national confirmed_case_history and confirmed_death_history. All 20 dates reconcile. - [province_death_history] added; [province_confirmed_history] regenerated and extended (now 20 vintages from 15 June, was 19 from 18 June). - _patch_death_increments + a second composition in bvd_patch_joint, with a tight death-ascertainment prior. - Registered in the CI patch fit and pinned by a test on the case/death gap: if that gap ever vanishes the deaths stop identifying anything and the split silently reverts to prior-driven. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
…drop-in Two changes, both needed for the patch model to BE the headline joint. 1. Vectorise the composition likelihoods. The stick-breaking issued a scalar `~` per (patch, vintage): with two compositions over 20 vintages that is 80 tilde statements. Each one puts DynamicPPL bookkeeping on the Mooncake tape, and 80 of them inflated the tape enough that the gradient COMPILE ran past an hour -- the last fit sat at 170% CPU for 74 minutes without reaching iteration 25. At that speed the CI patch fit would have hit GitHub's 6h job cap, which this repo has hit before. Within a vintage the patches are genuinely sequential (patch p's trial count is what patches 1..p-1 left behind), but ACROSS vintages they are independent, so the vintages vectorise. Flipping the loop to patches-outer and scoring all vintages in one product_distribution cuts 80 tilde statements to 4. 2. Surface the full bvd_joint deterministic set. The patch model is the headline joint, not a side analysis, so a patch chain must carry every quantity a single-patch chain does. It was missing 51 of them, including the whole expected_*_T set that `forecast_reported` reads -- so a one-week-ahead forecast could not be produced from a patch fit at all, and analysis.jl would have failed at render time rather than in a test. The observation submodels are identical to bvd_joint's, so every state was already present; only the latent-derived quantities differ and those come from the patch state. bvd_patch_joint now surfaces all 68. Pinned by a test that asserts every deterministic the forecast machinery reads is present and finite on a real chain. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
Exposes `chn_patch` in _setup.jl alongside `chn_joint`, and adds a spatial section to analysis.jl reporting what the patch model adds over the national fit: the per-province case/death split, the per-patch outbreak sizes and Rts, the relative case ascertainment, and a comparison of the patch headline C_T against the single-patch one. The patch fit is exposed ALONGSIDE the joint rather than silently replacing it. The patch model is a different generative story, so the national headline is a real check on it: if the spatial structure were distorting the national fit, the C_T comparison is where it shows. Swapping it in as the headline without that comparison visible would hide exactly the failure mode worth watching for. The section leads with why the split is hard -- a province's confirmed count is the product of its incidence and its case-finding, and only the product is observed -- and with why the deaths resolve it. It also flags the trap: a naive CFR comparison would overstate the ascertainment gap, because a fast-growing epidemic biases the observed CFR down and Ituri grows faster than Nord-Kivu. The model predicts that right-censoring from each province's own incidence curve rather than mistaking it for case-finding. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
The meta-population model becomes the headline `bvd_joint`, and the old single-population composer is deleted. They were never really two models. With `n_patches = 1` the patch machinery collapses exactly onto the single-population one: the sum-to-zero deviations vanish, a 1x1 kernel has nothing to couple, and with no province data there are no composition terms. Verified: bvd_joint and bvd_patch_joint(n_patches=1) sample the SAME 62 parameters, differing only in submodel prefixes. So `n_patches` is now a keyword defaulting to 1, and the single-patch case is fitted as the `sens_no_patches` sensitivity -- the check on the spatial structure. If the patch structure were distorting the national fit, the two C_T posteriors would part company, and the analysis puts them side by side. Making the collapse exact needed three fixes, each of which was a real defect: - the deviation machinery (sigma_level, sigma_delta, Omega_L, z_level, z_drift) was sampled even with one patch, where delta is identically zero -- five prior-only dimensions the likelihood never touches - seed_fraction was sampled with no secondary patch to seed - the recovered stream was not given the pooled dispersion, so it sampled its own, silently breaking the partial pooling the other streams share A guard now errors if per-province data is supplied with n_patches = 1. That is not hypothetical: the rename in this commit initially dropped the argument at every call site, which would have run the headline fit on a single patch holding the entire national total, with the spatial structure quietly gone and the fit looking perfectly healthy. Pinned by a test. Analysis: a Spatial structure section in Methods style -- what identifies the split, then the results. `patch_summary_table` now reports six quantities per location: cumulative infections, Rt, daily infections, the log-Rt deviation from the national trend, the log-Rt contrast against Ituri, and the relative case ascertainment. The last two are deliberately adjacent: the case composition identifies only their product, so a provincial Rt reported without the ascertainment beside it invites a case-finding artefact to be read as epidemiology. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
The headline is the meta-population model; n_patches = 1 collapses it exactly onto the single-population model the earlier releases used. That degenerate case is fitted as sens_no_patches and compared against the headline here. The patch model is a different generative story, so the national outbreak size is a genuine check on it rather than a restatement: if the spatial structure were distorting the national fit, the two C_T posteriors would part company. It is also a check on the identification. The per-province split rests on the confirmed deaths -- the case split alone cannot separate incidence from case-finding, because only their product is observed. With the patches off the provincial ascertainment cannot be estimated at all, and the national estimate is what remains. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
…ontract Both render jobs failed in CI with `KeyError: key :cumulative_infections not found`. The headline chain was missing it. The reason it slipped through is worth recording. When the patch composer took over as bvd_joint, I brought across its deterministics by diffing the two FUNCTION BODIES. But the single-population composer did not surface `cumulative_infections` from its own body -- it came from the `_latent` submodel it called. So the diff showed every other deterministic as missing and not that one, the tests passed, and the failure only appeared when the docs tried to render. Diffing bodies is not enough. The drop-in test now asserts against the keys that are actually READ: every chain key that src/ or docs/ pulls off a joint chain (30 of them), collected from the source rather than from what I happened to remember. A missing one now fails in the test suite instead of at render time. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
…cally
Three CI render failures in a row, none of which failed a single unit test.
All three were caught by rendering the docs locally, which was not previously
possible in reasonable time.
1. The Rt walk was attached UNPREFIXED, so its parameters reached the chain as
`sigma_rw` / `log_R0` / `z` rather than `rt_state.sigma_rw` etc. The docs
read the prefixed names. The same parameters are sampled either way, so a
parameter COUNT check passes and only a render fails -- which is exactly why
I dismissed this difference as cosmetic when the degenerate-case test showed
it. It was not cosmetic. Now attached prefixed, matching the convention the
rest of joint.jl documents.
2. `region_drift_sd` is a per-PATCH vector, and it was being passed to
summary_table, which is for scalars. It died deep inside `quantile` with a
cryptic `isfinite(::Vector)` MethodError. Moved to patch_summary_table,
where it belongs -- it is a per-location quantity ("is this province's Rt
pulling away from the national trend?"). summary_table now errors clearly on
a vector-valued key instead, naming the parameter and pointing at the right
table.
3. Added BVD_FIT_SAMPLES / BVD_FIT_CHAINS. A full render refits thirteen models
at 1000x2, which is hours, so the docs build could not be used as a check
that the pages still render -- and CI became the test loop. Now
`BVD_FIT_SAMPLES=60 task docs-main` exercises every code block on the page
(every chain key it reads, every table and plot call) in minutes. The draws
are useless for inference; that is not what is being tested.
Also merged main (SitRep 058, 11 July) and advanced the per-province data to
match: 21 vintages of cases and deaths, 21 of laboratory throughput, all
reconciling exactly with their national totals. Both scanners are gated, so a
mis-parse refuses to emit rather than silently writing bad data.
Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
Main lowered the joint fit from 0.95 to 0.90 but left the frozen joints and the sensitivity refit variant at 0.95. They are the same model on frozen or perturbed data, so the split looks like an oversight rather than a decision. All four fit paths (headline joint, sens_no_patches, the frozen joints and the refit variant) are now on 0.90. One setting, one sampler behaviour, and the frozen-vs-live comparisons are no longer confounded by a different acceptance target. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
The headline joint (n_patches=3) is the slowest fit in the matrix and hit GitHub's 6h job cap: at 1000 draws it ran ~5.8h before cancellation, which skipped the render jobs. This is NOT a mixing failure, and the fix is deliberately not a structural one. Investigation: - An Rt-parameterisation experiment (three alternatives via worktree subagents: independent per-patch walks, a rank-1 factor model, and a constant offset) found ALL of them equivalent in geometry to the current MVN+LKJ walk -- every one saturated the NUTS tree depth just as much. So the Rt deviation structure is not the cost, and the LKJ correlation / sum-to-zero coordinate I suspected are not a funnel. Each variant still recovered the provincial split (Nord-Kivu Rt ~ Ituri, infection share ~11-12%), so that result is robust to how Rt is parameterised. - Profiling the production chain by lag-1 autocorrelation found NO funnel: the worst-mixing parameter is 0.25 (recovery_delay_mean), everything else lower, and the worst offenders are in the base CFR/treatment model, not the patch structure. The chain mixes well; the deep trees are the price of an intrinsically high-curvature posterior that NUTS traverses effectively (few divergences, low autocorrelation, valid samples). So the principled lever is fewer DRAWS, not lower per-draw quality. At 800 draws the fit lands ~4.6h, comfortably under the cap, and with autocorrelation this low the effective sample size stays ample (~950+ for the worst parameter across 800x2 draws). Env-overridable via BVD_JOINT_SAMPLES; only the headline joint is affected -- the single-stream and frozen fits stay at 1000. Co-authored-by: Sam Abbott <contact@samabbott.co.uk>
The spatial section reported one table of every province stacked against every quantity, twenty-one rows deep, which is unreadable as a comparison. Split it: a cross-province overview with one row per province, then one table per province, then the hyperparameters. Add the figure the section was missing. Provincial Rt over time was not shown anywhere, only its value at the cut-off, so the question the patch model exists to answer could not be read off the page. `plot_rt_patches` facets one panel per province with the national trajectory behind it, rebuilt by `reconstruct_patch_rt` from the deviation knots the chain now carries as `delta_knots`. A test pins the reconstruction against the model's own `R_T_patch` at the cut-off, so the panels cannot drift away from the tables unnoticed.
The meta-population notes sat under a second `## v1.13.0` heading, a version already released, so the file carried the heading twice and the changes were attributed to a release that does not contain them. Move them to v1.14.0 and bump the version, following the convention every other feature change here uses.
Splitting the country into provinces adds no national data, so the national outbreak size should not move. The prose framed the n_patches = 1 comparison as a check that would part company if the spatial structure distorted the national fit, and read it as passing. On the last completed fit the two posteriors do part company, by about threefold, with non-overlapping 90% intervals. Say what the comparison is for and what disagreement means, so the reader draws the right conclusion from whatever the numbers turn out to be. Also correct the joint fit's draw budget. The constraint is the timeout-minutes: 350 on the fit job, not GitHub's 6h ceiling, and the 800-draw setting was cancelled at that mark without finishing, so the docs have never rendered on this branch.
a9f9bc0 to
e71f591
Compare
The spatial sensitivity reads a difference between n_patches = 1 and n_patches = 3 as evidence about the spatial structure. That is only valid if the two are otherwise comparable, and they are not. Two structural asymmetries inflate the national total as patches are added, both visible in the prior, before any data: - importation manufactures infections instead of moving them. The destination gains eps * K * I_prev and the origin loses nothing, so every patch's total rises and none falls. Worth ~1.31x. - the primary patch is seeded from growth_state.C_T, sized as if it carried the whole national outbreak, and each secondary adds a further fraction on top, so the national initial condition grows with the patch count. Worth ~1.33x. Together the prior-predictive national C_T is ~1.75x larger at three patches than at one. Same direction, and much of the size, as the gap between the fitted patch model and the single-population headline. Pinned as @test_broken so the invariants are stated and a fix flips them green, with the observable symptoms pinned as live tests.
…alone The registry comment, the release notes and the analysis intro each still said or implied that the national fit is undisturbed by the patch structure, which the prior-predictive comparison contradicts. Say what is actually the case in all three. Also cut repetition in the spatial section: the identification argument was spelled out under "Identifying the split" and restated under "Across provinces", and the importation coupling was introduced twice. Split two run-on sentences.
The two fits are the two halves of the spatial sensitivity, so a gap between their C_T posteriors is only evidence about the spatial structure if nothing else differs. They had drifted apart by nine keyword arguments. The worst is `genetic`, which defaults to `nothing`: the headline was running with no genetic TMRCA likelihood while the control had one, so the comparison was not controlled at all, and the headline had no constraint on outbreak age. Also absent: onset_curve_history, the three confirmed_break arguments, both in-care histories, background_re and confirmed_positivity_link. Both specs now splat one shared NamedTuple, with a second holding the patch arguments alone, so they cannot diverge again. Also pin the dominant inflation mechanism. The deviations are centred unweighted, so the trend the molecular clock constrains is the geometric mean of the patch Rts while the epidemic runs at the force-weighted arithmetic mean. Measured at the prior, the aggregate exceeds the trend by ~7.5% at the median, which compounds over the renewal window.
Splitting the country into provinces adds no national data, so the national trajectory must be identical to the single-population one given the same trend and the same total seed. It was not. Three structural asymmetries inflated it as patches were added. Aggregation, much the largest. The deviations are centred unweighted, so the trend the molecular clock constrains is the geometric mean of the provincial reproduction numbers while the epidemic runs at the force-weighted arithmetic mean. The renewal is now anchored: each day every province is scaled by one common factor so the implied national reproduction number is the trend exactly. The deviations become pure contrasts between provinces and the national level is left to the trend alone, which is what the model already claimed. Ratios between provinces, which is what the composition data identify, are untouched. Importation manufactured infections rather than moving them, crediting the destination while the origin lost nothing. It now debits the origin exactly what the destinations are credited. The seed fractions added to the national cryptic seed rather than partitioning it, so the initial condition grew with the patch count and 2^m was silently redefined from the country's cryptic size to Ituri's. They now partition it. With the same trend and the same total seed, three provinces reproduce the single-population trajectory to machine precision, day by day. On the test setup the old code inflated the national cumulative total roughly thirtyfold. Pinned deterministically: a Monte Carlo comparison of prior medians cannot resolve this, since the prior on C_T is heavy-tailed enough that the median is dominated by sampling noise.
Anchoring scales every province by one common factor each day, so the realised provincial reproduction number is the walk times the deviation times that factor. `reconstruct_patch_rt` rebuilt only the first two, so the faceted figure sat on a different scale from the `R_T_patch` the tables report. The factor depends on how the force is split across provinces, so it cannot be recovered from the deviation knots. Carry it on the chain as `rt_anchor_scale`, one value per day, and apply it when rebuilding. Caught by the test that pins the reconstruction against the model's own `R_T_patch`, which is what that test is for.
Three provinces cost about twice as much per draw as one. Measured, the gradient is 26.2 ms at one patch over 140 parameters against 53.0 ms at three over 220, a factor of 2.03, and the extra dimensions lengthen NUTS trajectories on top of that. The single-patch fit takes 314 minutes at 500 draws, so three patches at 500 needs on the order of 700, against the 350-minute job budget. Both 800 and 500 draws were cancelled at that mark without finishing, so the documentation has never rendered on this branch. 200 draws lands near 280 minutes. It is half what every other fit uses, which is a real cost to the headline: the alternatives are a runner without a 6h ceiling or a cheaper patch model, and raising the timeout is not one of them since the ceiling above it is 360.
The cost is in trajectory length, not the gradient. At 0.90 the joint runs a median NUTS tree depth of 9 with 43% of draws at the cap of 10, so each draw spends 512 to 1024 leapfrog steps. A larger step size shortens those geometrically, which is the only lever here with a factor in it. The model side was measured and is null, consistent with what earlier speedup work on this repo found: the per-province convolutions read only 147 of 173 days and are a small part of the density, so truncating them buys a percent or two on a 53 ms gradient. The headline and its spatial control share one setting through joint_target_accept, so they cannot drift apart the way the model keywords did. Adapt delta changes sampling efficiency rather than the target posterior, but the pair is only comparable if both move together.
Two things the first successful patch fit surfaced. The treatment submodel was called without conf_hazard_daily, which the single-population composer passes. That sets split_active = false, so the Tableau 6 confirmed and suspect in-care split silently drops out of the likelihood and incare_confirm_modifier is never sampled. The analysis page reads that deterministic and died on a KeyError, which is the first time anything caught it: a parameter count cannot see a missing likelihood term, and the page had never rendered from a patch chain. Adapt delta goes back to 0.90. Dropping it to 0.80 was meant to shorten trajectories, since the cost is trajectory length rather than the gradient. Measured, it did the opposite: tree depth pinned at the cap of 10 for every draw with 1023 leapfrog steps and an adapted step size of 0.003, against a documented median depth of 9 at 0.90. The 178-minute wall clock came from cutting draws to 200, not from the adapt delta. Recorded in the registry so the next person does not retry it.
Makes the headline model a meta-population: a renewal equation per province (Ituri, Nord-Kivu, Sud-Kivu) coupled by importation, with every national stream fitted against the summed provinces.
n_patchesdefaults to 1, which collapses it onto the previous single-population model, so there is one model rather than two. Adds the per-province data to fit it.What was wrong, and the fix
The national outbreak size depended on how many provinces the country was split into. Splitting adds no national data, so it must not. Like-for-like on 13 July the patch fit gave C_T 13,213 [10,390–18,963] against main's ~4,700 [3,455–7,051] — non-overlapping 90% intervals.
Three structural asymmetries caused it, all visible in the prior, before any data. All three are fixed.
1. Aggregation, much the largest. The deviations are centred unweighted, so
mu(t)— the trend the molecular clock constrains — is the geometric mean of the provincial reproduction numbers, while the epidemic runs at the force-weighted arithmetic mean thatimplied_national_Rtrecovers and the chain reports asR_T. Arithmetic exceeds geometric, so the country grew faster than its own trend, and the excess was spent in the pre-surveillance window where no counts contradict it. Measured at the prior, the aggregate exceeded the trend by 7.5% at the median — exactly the observed posterior shift,R_T1.22 → 1.31, a ratio of 1.074.The sign was forced by the data: Ituri carries ~90% of the force and Sud-Kivu is frozen at 3 confirmed cases across every vintage, so the compositions push the small provinces below trend and unweighted centring pushes Ituri above it.
The renewal is now anchored: each day every province is scaled by one common factor so the implied national reproduction number is
mu(t)exactly. The deviations become pure contrasts and the national level is left tomualone, which is what the model already claimed. The ratios between provinces, which is what the composition data identify, are untouched.2. Importation manufactured infections rather than moving them. The destination gained
eps * K * I_prevand the origin lost nothing, so every patch's total rose and none fell. It now debits the origin exactly what the destinations are credited.3. Seeding scaled with the patch count. The fractions added to the national cryptic seed instead of partitioning it, silently redefining
2^mfrom the country's cryptic size — which is what themprior is elicited as — to Ituri's. They now partition it.Result. With the same trend and the same total seed, three provinces reproduce the single-population national trajectory to machine precision, day by day, with importation live and heterogeneous deviations. On the test setup the old code inflated the national cumulative total roughly thirtyfold; it is now exact.
The invariant is pinned deterministically rather than by comparing prior medians — the prior on
C_Tis heavy-tailed enough that its median is dominated by sampling noise, which is why this was not caught earlier.The comparison was also not controlled. The headline fit and
sens_no_patchesare the two halves of this sensitivity, so they must differ only in the patch structure. They had drifted apart by nine keyword arguments.geneticdefaults tonothing, so the headline was silently running with no genetic TMRCA likelihood while the control had one. Measured, that term is a soft one-sided penalty: nonzero on 31 of 40 prior draws, up to ~1.1 log units against a total log-density of ~12,100 — necessary for the comparison to be controlled, but weak, and not on its own an explanation for the gap. Both specs now splat one shared configuration.Still open: the fit does not fit the CI budget
Three provinces cost about twice as much per draw as one. Measured:
That is a factor of 2.03, and the extra dimensions lengthen NUTS trajectories on top of it. The single-patch fit takes 314 minutes at 500 draws, so three patches at 500 needs on the order of 700, against the
timeout-minutes: 350budget on the fit job. Both 800 and 500 draws were duly cancelled at that mark without finishing, so the documentation has never rendered on this branch.Set to 200 draws, which should land near 280 minutes. That is half what every other fit in the matrix uses, so the headline would have half the effective sample size of its own sensitivities. Raising the timeout is not an option — GitHub's ceiling is 360. The real alternatives are a runner without that ceiling, or a cheaper patch model. Worth a decision rather than my quiet choice.
Consequently no posterior fit of the corrected model exists yet. Every number above is deterministic or prior-predictive. The control to compare against is
sens_no_patchesat C_T 8,625 [6,391–13,126], which is unaffected by the fixes: at one patch the anchoring factor is exactly 1.0 on every day of every draw and no seed fraction is sampled, so that path is bit-identical to before.The model
Provincial reproduction numbers are the national weekly-knot walk plus deviations that sum to zero, drawn from a correlated random walk:
mu(t)is the existing national walk and is the pooling target.sigma_deltais sampled, so whether the provinces share one temporal shape is estimated rather than assumed;sigma_delta -> 0recovers a fixed ratio between them. The nationalR_Tis backed out by inverting the renewal equation on the summed infections, which is the force-of-infection-weighted mean of the provincialRts.The split is identified by the deaths. A province's case count is the product of its incidence and its case-finding, and only the product is observed; the tests-analysed denominator cancels out of the normalised shares. The case-fatality ratio and the death-confirmation probability belong to the virus and to a national laboratory rather than to a province, so they cancel out of the normalised death shares. The death split identifies the incidence split and the case split then identifies ascertainment as the residual. Both are scored as compositions conditional on the national total, so neither re-scores data the national streams already carry.
Connectivity is a structural assumption. There is no mobility data, so the kernel is a fixed weighting by destination population with a sampled intensity, weakly identified against the secondary-province seeds.
Data
[province_confirmed_history],[province_death_history]— per-province cumulative confirmed cases and deaths from Tableau 1 (20 vintages), scanned byscripts/scan_province_tableau1.jl.[province_lab_daily_history]— per-province laboratory throughput (18 vintages), scanned byscripts/scan_province_lab.jl.Both scans are gated on reconciling exactly with the national totals on every date.
Reporting
The spatial section previously stacked every province against every quantity in one twenty-one-row table, and never showed provincial
Rtover time — the question the model exists to answer. Now:The trajectory is rebuilt by
reconstruct_patch_rtfrom the deviation knots the chain now carries asdelta_knots, and a test pins that reconstruction against the model's ownR_T_patchso the panels cannot drift from the tables.Other changes in this round
## v1.13.0heading, a version already released. Moved to v1.14.0 and bumped the version.Review
The anchoring is the change worth arguing about.
test/test_patch_model.jlexplicitly rejected reference coding (delta_1 = 0) as an identifiability artefact and chose unweighted centring instead; anchoring keeps the sum-to-zero deviations exactly as they were and fixes the aggregation at the renewal instead, so no province is privileged andmurecovers its stated meaning. If you would rather fix it by centring on incidence weights, that is the alternative and it is a smaller change to the maths but a larger one to the interpretation ofdelta.This was opened by a bot. Please ping @seabbs for any questions.