Repository navigation
Conversation
…hen an index is unsampled records_in_range now probes tidesdb_range_stats over [value, successor(value)) for a point equality on a non-unique index with no cached sample, so a never-analyzed low-cardinality index estimates correctly from metadata instead of the records/10 fallback that reads one row per value and drives a full scan. unique, primary-key, and already-sampled indexes keep the cheap cached rec_per_key, so the sampled path is unchanged and points/ranges do not regress. adds rir_prefix_successor, updates the optimizer doc and the index_stats test to the improved before-analyze estimate.
a multi-row DELETE that removes every row across a contiguous primary-key span now writes one range tombstone instead of a per-row tombstone for each key. delete_row buffers each primary-row data key and holds its tombstone back; end_bulk_delete counts the live rows thetouched [min, max] span still holds and, when that equals the buffered count so the statement removed the whole span, issues one tidesdb_txn_delete_range. a survivor from aresidual filter, an IN-list gap, an ICP rejection, or an ORDER BY LIMIT leaves more live rows than buffered keys and falls back to per-row tombstones, so the result never changes. advertises HA_CAN_FORCE_BULK_DELETE so the server drives start_bulk_delete for a single-table delete, which also activates the existing compact-after-range-delete path. deferral is skipped for a table with a delete trigger and under galera, rows than buffered keys and falls back to per-row tombstones, so the result never changes. advertises HA_CAN_FORCE_BULK_DELETE so the server drives start_bulk_delete for a single-table delete, which also activates the existing compact-after-range-delete path. deferral is skipped for a table with a delete trigger and under galera, secondary-index entries are still deleted per row, a delete past a cap flushes to per-row tombstones, and the range tombstone carries the library's own OCC conflict detection. on a pk-only table a 200k-row range delete drops from ~0.33s to 0.192s and from 2.6MB of tombstone writes to essentially none. adds tidesdb_range_delete covering clean ranges, gaps, residual filters, LIMIT, rollback, secondary indexes, and triggers.
the analyze table diagnostic prints a write-amplification note whose wal byte count and derived ratio drift run to run from wal framing, so any test that records analyze output flakes under heavy parallel load.tidesdb_index_stats and tidesdb_analyze already normalize these figures with replace_regex, but tidesdb_row_cardinality recorded them verbatim and so failed intermittently on a wal= mismatch.apply the same mask before each analyze and keep asserting the real row counts through information_schema.TABLES, which is what the test is actually about. tidesdb_tpcc_contention, flagged alongside this, was investigated and left unchanged.its retry loop, its 1180/1213/1205 conflict handler, and the commit path are all correct, and no lost or double increment reproduced across roughly six hundred executions including two hundred under amplified sixteen-way contention with forced memtable backpressure, so its single earlier failure reads as an environmental blip rather than a logic defect.
the neword_incr retry loop gave up an increment under disk pressure.a losing optimistic attempt rolls back without an fsync, so it can spin through the whole thousand-try bound while the winning connections are still fsyncing their commits under full sync mode, then exit having never committed.the district counter then lands one short and the test fails on a 5004 against 5005. add a short randomized sleep after a conflicting attempt rolls back, the same way a real tpc-c client backs off, so the contention drains before the next try and every increment lands.the asserted counts are unchanged.reproduced reliably by running the storm under background disk io load, where an unfixed sixteen-way amplifier failed a hundred of a hundred and twenty runs and the backoff took it to a hundred and twenty of a hundred and twenty.
… for mariadb pick up
…fault table creation may fail with warning, this i documented
the plugin compiled cleanly only against 12.3 and 13; older servers drifted in ways the existing version guards did not cover.it now builds and passes the tidesdb suite on 11.0.1, 11.8.8, 12.0.1, 12.3.2, and 13.0.1. st_typelib carries a fifth hidden-values member on 11.8 and on 12.2 and up, but not on 11.0 through 11.7 or on 12.0 and 12.1, a boundary that is not monotonic across the 11 and 12 branches, so a macro supplies the fifth initializer exactly where the member exists rather than over-initializing the aggregate. handler::referenced_by_foreign_key returned uint and was non-const before 11.7 where it became bool const noexcept, and init_tmp_table_share gained a trailing thread_specific argument in 11.7, so both are guarded at 11.7.commit_ordered lost its leading handlerton argument in 11.8 like its sibling hooks but had been left unguarded, so it now follows the same version split.XID::length was non-const on the older servers, so tdb_xid_key casts away const to bind whichever declaration the linked server provides. the prefix-index test recorded a max_sort_length note the server emits from 11.8 on but not earlier, so the test suppresses it to stay stable across versions. verified by building each server and running mtr
…ervers mariadb 11.4.13 introduced MARIADB_ADD_PLUGIN, which takes version, author and description metadata and drives a new packaging and distribution model, and made MYSQL_ADD_PLUGIN an alias for it.older servers ship MYSQL_ADD_PLUGIN alone and it does not accept the metadata, so call the richer macro only where it exists and fall back to the plain call everywhere else, the same call that already builds across 11.0 through 13. the declaration now reads PLUGIN_AUTHOR, PLUGIN_DESCRIPTION, PLUGIN_LICENSE and PLUGIN_VERSION from that metadata, with #ifndef fallbacks so it still compiles on servers that define none of them.it keeps TIDESQL_VERSION_HEX for the hex version rather than the macro's PLUGIN_HEX_VERSION, which expands from server-side V_MAJOR and V_MINOR variables that some plugin.cmake releases leave unset, and it drops the maturity to beta since a first packaged release must be below gamma. when no system library is found the build now fetches tidesdb v10.0.0 and links it statically through an ExternalProject, so the plugin can be built with no run-time dynamic dependency and stays pinned to its library version. two api shapes are backported into maintenance releases at unpredictable patch levels, so version thresholds do not track them.handler::referenced_by_foreign_key became bool const in the mainline at 11.7 but is bool in 11.4 too, so guard it at 11.4.st_typelib's fifth member lands in some maintenance releases and not others, so initialize the four members every release carries and suppress the missing-field warning rather than guess a boundary. MDEV-39776 quotes engine option values in SHOW CREATE and CREATE_OPTIONS on the new servers, so the five tests that read them normalize the quoting, through a replace_regex on the create output and a quote-stripping REPLACE on the option probes, leaving one recorded result that matches quoted and unquoted servers alike. verified building and passing the tidesdb suite on 11.0.1, on 13, and on the new 11.4.13, with no plugin warnings on any of them.
make_comparable_key gives a nullable key part a fixed width of one null indicator byte plus kp->length zeroed value bytes, so every part after it sits at a predictable offset. both sort-key decoders, the covering-index reader in keyread_decode_key_parts and the index-condition-pushdown reader, advanced past a null part by only the indicator byte and never skipped the value bytes, so every later part was decoded from the wrong offset. a column after a null indexed column came back null or as whatever the misaligned bytes decoded to, and a pushed predicate was evaluated against the wrong column, both silently with no error or warning. the base row is stored correctly, only these two key-decode paths were wrong, so it only showed when the optimizer picked a covering scan or pushdown rather than reading the row. skip the full one plus kp->length width at both sites, bounds-checked against the key end. the descending case stores the indicator inverted but keeps the same width, so the one fix covers it. adds tidesdb_null_keypart_decode, which forces the index and selects only covered columns to exercise both readers on ascending, two-parts-after-null, and descending indexes. verified building green and passing on 11.0.1, 11.8.8, 12.0.1, 11.4.13 and the 13 main build
the plugin now declares MariaDB_PLUGIN_MATURITY_BETA, and plugin_maturity in the server config is the minimum maturity it will load. left at gamma the server refuses the plugin, since beta ranks below gamma. set it to beta in the install.sh config so ha_tidesdb loads at startup as intended.
…s against the code an ALTER that only flips the ENCRYPTED table option arrived as a create-option change, which check_if_supported_inplace_alter classified as instant, so the engine set share->encrypted without rewriting any rows. the rows already in the table kept their old on-disk format, plaintext after ENCRYPTED=YES or ciphertext after ENCRYPTED=NO, while the read path now treated them as the other, corrupting access to them. force the copy algorithm when the encrypted flag changes, the same way fulltext and spatial indexes already force it, so every row is rewritten through write_row and re-encrypted or decrypted to match. verified on mariadb 13.1.0 with a file-key-management provider, the rows stay readable and identical across ENCRYPTED=YES and back, ALGORITHM=INSTANT is refused for the change, and a non-encryption instant ALTER still works. the full tidesdb mtr suite green and mean. the change came out of code review, install.md carried an old SHOW ENGINES comment and engine name and omitted the plugin-maturity=beta a beta plugin needs to load on a stable server. system-variables.md and backup.md called the checkpoint a near-instant hard-link snapshot when it is a durable flush of the wal, value log, and manifest followed by a byte-for-byte copy. maintenance.md said REPAIR drops corrupt entries, where a bad block fails the repair instead. monitoring.md left the value-log device out of the write-amplification and io-device sections. encryption.md is reworded now that the ALTER above rewrites the rows. three user-facing engine strings were wrong the same way and are corrected too, the memtable_sync_mode NONE description which said it relies on the os page cache when it stages in the in-process ring, the checkpoint_dir description which called it a hard-link snapshot, and a stale foreign-key header comment that still said cascade and set-null fall back to restrict.
tidesdb_rpl.tidesdb_rpl_types recorded a 'key too long, max 255 bytes' note on the CREATE TABLE because VARCHAR(64) is 256 bytes under utf8mb4 and trips the handler's default 255-byte key-part limit. that note only fires when the server default charset is utf8mb4, so the test passed where the result was recorded (12.x/13.x, utf8mb4 default) and failed on 11.4 (latin1 default), where the column is 64 bytes and no note appears. pin DEFAULT CHARSET=utf8mb4 on the table so the column is 256 bytes and the note is emitted the same way on every server default. the recorded result keeps the note and now matches regardless of version
the galera participation is gated on WITH_WSREP, which an in-tree build gets from my_config.h. an out-of-tree build, like the mariadb foundry packaging build, compiles against the installed my_config.h, which omits WITH_WSREP, so the whole galera path compiled out. the handlerton never advertised HTON_WSREP_REPLICATION, and a tidesdb table replicated its ddl through the provider but not its rows, diverging the cluster. define WITH_WSREP in ha_tidesdb.h when it is not already set and the wsrep-lib header wsrep/client_service.hpp is on the include path. that header is the real gate, because with WITH_WSREP on the server's own sql_class.h pulls it in, so keying on it turns galera on exactly where it can compile and leaves it off against a distribution that does not ship the wsrep headers, which then builds clean without galera. also make tidesdb_galera portable across the server default charset by normalizing the charset and collation clause in show create table, which varies from latin1 on older servers to the uca1400 collations on newer
…makelist patch; update doc; a PLUGIN_VAR_MEMALLOC string sysvars update callback is handed the new value in a temporary buffer that the server frees once the callback returns, so it must take an owned copy the way the framework's own update_func_str does. four of the plugins callbacks stored the temporary directly, leaving the variable pointing at freed memory. it read back empty on 64-bit and, where the freed slot had been reused, as an unrelated server string on 32-bit, which is how tidesdb_fts_blend_chars showed SESSION_VARIABLES in its value column on the 32-bit ci. add a helper that copies the value and frees the previous one, and use it in the blend-chars, stopword-table, backup-dir and checkpoint-dir update callbacks. the blend-chars test now sees the value it set, so its result records that instead of the empty string the bug produced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.