The file group reader and the log and buffer code it reaches take a HoodieTableMetaClient but use only the table config, the base path, a storage factory and, in two places, the timeline. Those two places (the committed-instant check on log blocks of tables before version 8, and the schema-history lookup for schema-on-read) load the timeline on the executor, so every task repeats that .hoodie I/O and tasks of one query can see different timelines.
Proposal: pass the reader a small table state with the table config, the base path, and the committed instants and schema history captured on the driver where the timeline is already loaded.
part of #20064
PR: #20078
The file group reader and the log and buffer code it reaches take a
HoodieTableMetaClientbut use only the table config, the base path, a storage factory and, in two places, the timeline. Those two places (the committed-instant check on log blocks of tables before version 8, and the schema-history lookup for schema-on-read) load the timeline on the executor, so every task repeats that.hoodieI/O and tasks of one query can see different timelines.Proposal: pass the reader a small table state with the table config, the base path, and the committed instants and schema history captured on the driver where the timeline is already loaded.
part of #20064
PR: #20078