Repository navigation
Compute window std dev, skew and kurtosis together - #558
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The shared reduction changes standard-deviation results for windows containing infinities from NaN to zero or infinity, altering feature values and requiring a fix with infinity-case equivalence tests.
1 open issue
What changed in this PR
This PR reduces window-feature computation time by sharing intermediate calculations for standard deviation, skew, and kurtosis in JABS feature extraction.
Changes:
- Adds a combined window-statistics function and reusable central-moment helpers.
- Routes default moment operations through the shared calculation while preserving subclass overrides and output order.
- Adds equivalence tests for combined statistics and feature-level integration.
| File | Description |
|---|---|
tests/feature_extraction/window_operations/test_window_stats.py |
Tests combined statistics against separate operations. |
tests/feature_extraction/test_feature_base_class.py |
Tests output equivalence, operation order, and subclass overrides. |
src/jabs/feature_extraction/window_operations/window_stats.py |
Adds shared moment calculations and the combined statistics function. |
src/jabs/feature_extraction/feature_base_class.py |
Integrates shared calculations into standard window features. |
Review effort: Balanced
| # equal to np.nanstd: both are the square root of the same biased variance | ||
| std_dev = np.sqrt(variance) |
There was a problem hiding this comment.
Good catch — fixed in 02c2b76. The std dev now sums the squared deviations like np.nanstd: only values that were nan in the input are zeroed, so a nan from inf - inf still propagates (skew/kurtosis keep their original nansum behavior, which was already shared). Added all-inf, mixed ±inf, and inf+nan series to the equivalence test at three window sizes (the previous commit fails all 9 cases); the 54,936-array comparison against main on the sample pose files is still exact.

Summary
Window features are now the largest share of feature compute time (~59% in a recent
jabs-init --timing-reportrun). Profiling one identity showed thestd_dev,skew, andkurtosiswindow operations together taking ~28% of window time, and each independently recomputes the same per-window count, mean, centered values, and variance for every column.window_stats.window_std_skew_kurtosis(values, window)computes all three from one set of shared terms (_central_moment_terms).np_skewandnp_kurtosisnow use the same helpers, so the arithmetic is shared rather than duplicated. The standard deviation issqrtof the same biased variancenp.nanstdcomputes.Feature._window_standardcomputes the three through it in one pass (_shared_moment_features), keeping the original operation order. An operation is only routed through the shared path if it still uses its default implementation, so a subclass that overrides one of them keeps its own.window_std_dev/window_skew/window_kurtosisfunctions are unchanged.Performance
tests/data/sample_pose_est_v6.h5,get_window_features(10)for all 4 identities, best of 3:Feature values
Unchanged — no
FEATURE_VERSIONbump. Verified by dumping every window feature (54,936 arrays) for every identity in the v5 and v6 sample pose files at window sizes 5 and 10 onmainand on this branch, and comparing for exact equality including key order.Testing
window_std_skew_kurtosisis bit-identical to the three separate ops on the existing test series (scattered nan, nan runs wider than the window, constant, all-nan, mostly zero) at three window sizes.Feature.windowresults equal each window op run on its own, in op order; a subclass overridingskewkeeps its implementation.uv run pytest --ignore=tests/ui/test_viewer_window.py: 2090 passed, 7 skipped.tests/ui/test_viewer_window.pyaborts the run with SIGTRAP aftertest_video_without_pose_file_opens_without_overlayslocally onmaintoo, so it's unrelated.