Assign submitted rows to modeling tasks without expanding the grid

[annakrystalli]

Resolves #355.

What this changes

match_tbl_to_model_task() finds the modeling task that each submitted row belongs to.

Before, it built the grid of every value combination the config allows, then joined the data to that grid. Now it checks each column separately. A row belongs to a modeling task when every one of its values is allowed by that modeling task for that column.

The grid is never built. Memory now depends on the size of the submitted file, not on the size of the config.

Measurements

G1, G2 and G3 all submit about 65,000 rows. They differ only in what the config allows, which gives grids of 11.7M, 38.9M and 77.7M rows. The table shows peak memory and time, before and after.

check	size	before	after
match_tbl_to_model_task	G1	3,568 MB / 8.59s	221 MB / 0.05s
	G2	9,783 MB / 27.67s	210 MB / 0.04s
	G3	11,452 MB / 60.14s	211 MB / 0.05s
check_tbl_value_col	G1	3,579 MB / 8.71s	228 MB / 0.08s
	G2	9,925 MB / 28.70s	229 MB / 0.08s
	G3	13,424 MB / 59.64s	222 MB / 0.08s

No other check changed by more than the normal variation between runs. Rounds with more modeling tasks are also faster than before. At size M with 7 modeling tasks, matching goes from 1.07s to 0.71s, and check_tbl_value_col() from 1.20s to 0.80s.

Guide to the files

R/assign_tbl_to_model_task.R is new and does the work.

• get_config_mt_value_sets() reads the config once. For each modeling task, it returns the values allowed in each column.
• assign_mt_rows() selects the rows. It first uses match() on output_type_id to find the position of each row in the config list for its output type. A row with no position is not accepted by this modeling task. It then checks each task ID column with %in%.

One point to note while reading this. The submitted data is never converted. %in% calls match(), and match() converts both sides to a common type. That type is character if either side is character. The submitted data is already character, so only the config values are converted. The config holds a few values and the data holds millions of rows, so this is important. This is also why tbl must be character.

R/match_tbl_to_model_task.R has the same arguments as before and calls the new function. join_tbl_to_model_task() is unchanged. check_tbl_values_required() still uses it, and #357 will replace it in a separate PR.

R/expand_model_out_grid.R: the part that reads the config moved to extract_round_property_values(). Both expand_model_out_grid() and get_config_mt_value_sets() call it, so they always read the config in the same way. The part that builds the grid is unchanged.

R/validate_model_data.R now gives check_tbl_value_col() the character table tbl_chr instead of the typed table.

Why check_tbl_value_col() now uses tbl_chr

It was the only data check given the typed table, and it converted every column except value to character itself. Each run therefore built the character version twice. The exception for value was pointless, because the check converts value to the type the config declares anyway.

It now uses the shared character table. This is not a speed change (about 5 ms on 460,000 rows); a run simply holds one character copy instead of two. Output is unchanged for the same input.

Changes in behaviour

These four are listed in NEWS under Breaking Changes.

all_character is removed. It controlled whether the grid was built as character, so that the grid and the data had the same types and could be joined. There is no join now, and tbl is always character, so the argument has nothing left to control.

Rows are returned in the order they were submitted. Before, the join returned them in config order. check_tbl_value_col_ascending() needs that order, because it calls diff(value) on the rows in the order it receives them. Sorting the output_type_id column does not work, because pmf categories are not always in alphabetical order (#78). The new argument order_by_config = TRUE returns rows in config order. check_tbl_value_col_ascending() now asks for it, and other checks can use it if they need the same thing.

This order is not identical to the old one. Task IDs are now sorted in config order, while expand.grid() varied the first column fastest. Inside each group of task ID values, the order of output_type_id is the same as before, and that is the only part the ascending check uses.

check_tbl_value_col() requires a character tbl.

A missing task ID column is now an error. Before, the grid supplied the missing column, and the join returned a number of rows that did not correspond to the submission. For example, removing location from a submission of 1,590 rows returned 450 rows and a dplyr warning about a many-to-many join.

Derived task IDs are no longer used for matching. They were used before, but only NA against NA: reading the config into value sets sets them to NA, and callers set the column to NA too. The result was always true. Skipping them saves one pass over the data per derived task ID. check_tbl_value_col_ascending() still sets them to NA, because it groups by these columns.

Other changes in this PR

• validate_derived_task_ids() has a new call argument, so its warning names the function the user called.
• _benchmark/check-calls.R now gives check_tbl_value_col the character table, like validate_model_data(). With the typed table the check still works, but match() converts a Date column once per modeling task: 3.81s instead of 0.68s at size M with 7 modeling tasks.
• New HUBVALIDATIONS_BENCHMARK_CHECKS variable, to run only named checks.

Tests

helper-match-tbl.R keeps match_via_grid(), which is the old implementation. The tests check that the new code puts the same rows in the same modeling tasks, for all 16 hubs that the test suite has a submission for. These cover schema versions v3 to v6, samples, derived task IDs, and rounds with several modeling tasks. Each hub is tested for every output type, and for both values of subset_to_tbl_cols.

The order is tested separately, using a config where the task IDs vary independently.

The test file runs in 5.4 seconds.

Not in this PR

check_tbl_values() (#356), check_tbl_values_required() (#357) and the sample checks (#368) still build the grid. This is why they are unchanged in the table. submission_tmpl() needs the full grid, because its purpose is to generate every valid combination.

[github-actions]

🚀 Deployed on https://6a842372c06bd99036b72a35--hubvalidations-pr-previews.netlify.app

[micokoch]

This looks great. I went through it carefully and was able to follow the narrative and logic. I had a couple of questions/comments, but I asked Claude, who explained them and they made sense. So, there are no special comments from me, but here are Claude's close reading comments:

Gave this a proper read (algorithm, the four breaking changes, both consumer
checks, the grid refactor, and the benchmark write-up) — it's solid, approving.
The core is equivalent to the old grid-join by construction: %in%-per-column
tests the same set membership without materialising the Cartesian product. I
traced the subtle bits and they hold:

• Unused/omitted task IDs become NA, and NA %in% NA is TRUE, so a modeling task
  only accepts rows that are NA there — exactly what the grid's NA column did.
• Point-estimate output types (output_type_id = NA) match via match(NA, NA).
• Derived task IDs are correctly excluded from both matching and ordering (safe:
  a derived value is a function of the matched task IDs, so it can't break a tie
  they didn't).
• The ascending check stays correct: check_values_ascending() re-groups with
  group_by(), which preserves within-group order, so even though the new sort
  makes task-ID groups non-contiguous, each group is still output_type_id-
  ascending.
  The equivalence test against match_via_grid() across all 16 fixtures is exactly
  the right way to prove a refactor like this, and lifting the config-reading into
  a shared extract_round_property_values() means the grid and set paths can't drift
  apart. Perf numbers check out against peak-results.csv, and G3 is now actually
  measured (it was defined-but-unmeasured in #360).
  One question (non-blocking) on coercion parity, and a few small nits below.

[LucieContamin]

Thank you for the update, it generally looks great and happy to see that the performance was increased.

I just wanted to check that the new match system keeps only row that are expected from the config files, the output is only used in check to test the variables types and value increase and not the content of the model output (test if its missing some specific value, like a specific target for a location, etc.). Am I correct?

Also, I request changes, because I have some question about some code, but mainly I really struggle with the documentation. It took me a long time to read all of it and try to make sense of some part of it. If it is only me, maybe ignore but I think it will be good to clarify some points, even for the internal documentation.

[annakrystalli]

Thanks for the thorough and careful review @LucieContamin !

On your first question: yes, and none of that changes here.

Matching returns only rows whose values the config allows and drops the rest, exactly as the grid join did. Its consumers are check_tbl_value_col() and check_tbl_value_col_ascending(), neither of which asks whether anything is missing. The checks that do test for missing or unexpected content, check_tbl_values() and check_tbl_values_required(), do not use it — they still build the grid, and #356 and #357 cover them.

One thing worth separating out, because it also came up on the output_types argument: validating one output type at a time is not new, and not something this PR introduces. expand_model_out_grid() has taken an output_types argument for a long time, and check_tbl_value_col_ascending() has looped over the output types present in a submission since January 2025 (b021028). It is not confined to the checks touched here, either — check_tbl_values() splits the submission by output_type, and check_tbl_values_required() maps over output types deliberately, to reduce memory pressure and to keep their evaluation separate (#177). Those two are where this approach goes next, in #356 and #357, so the pattern carries straight over. A modeling task can certainly define several output types; the checks just ask about one at a time.

On the documentation: that was fair, and it was not only you. I have been through every comment; 6cafdd9 has the rewrites. Thanks for being specific about which passages did not land, it made them easy to find.

[LucieContamin]

Thank you very much for the update and the additional information. It all looks good to me! I made some minor suggestion, feel free to ignore.

[LucieContamin]

Very minor note, config is alone on the line.

[LucieContamin]

Minor suggestion: the "Both" is not clear until we read the second sentence.

[LucieContamin]

Suggested change

	#' itself. `pmf` categories show why that matters: `"low"`, `"moderate"` and
	#' `"high"` have no useful alphabetical order, but the config lists them in the
	#' itself. Example of `pmf` categories show why that matters: `"low"`, `"moderate"` and
	#' `"high"` have no useful alphabetical order, but the config lists them in the

Minor suggestion