feat: support UUID document ids#4681
Open
sanikolaev wants to merge 15 commits into
Open
Conversation
Add UUID as a document id type backed by an internal numeric DocID_t surrogate and hidden UUID string storage. Related issue #1429
Use the schema @id locator when MatchIterator sorts and dedupes merged result sets. UUID agent replies append the internal numeric @id dynamically, so reading row offset 0 can collapse distinct remote UUID rows during distributed HTTP/JSON result merging. Test: php test/ubertest.php t --no-demo --strict-verbose --cwd -b /Users/sn/manticore_uuid/build/src/ -t /tmp/mcore-test517.JX5ycr/ -tt /tmp/mcore-test517.JX5ycr/ 517
Relax the UUID id DDL gate to allow sharded wrapper tables while still rejecting unsupported non-RT table types. Extend test_517 to cover sharded UUID create/describe/show, explicit and generated inserts, duplicate rejection, replace, update, delete, SQL reads, and HTTP search by UUID id.
Support UUID primary ids across DML/API boundary cases by keeping the hidden UUID storage attr reserved, validating JSON/bulk numeric ids on UUID tables, preserving transaction-local UUID to DocID mappings, and ensuring delete/update collectors keep internal numeric docids available while still allowing public UUID filters.\n\nExtend UUID id coverage in test_517 for transaction duplicates, hidden @uuid_id DDL rejection, JSON/bulk validation, and rowwise/columnar JSON delete paths.
Reject internal @id/@uuid_id access consistently for UUID-id tables, preserve public UUID id behavior, document the UUID-shaped id contract, and extend the focused UUID regression coverage for REPLACE, distributed, JSON, ES, and columnar paths.
Detect UUID primary ids for ES bulk writes to shard tables, reject user-defined @id on UUID-id tables, and extend test_517 coverage for the shard bulk and DDL guardrails.
Validate UUID primary ids with version/variant checks, map UUID public-boundary validation failures to HTTP 400, reject internal id projections through ES fields/stored_fields, and keep aggregate errors on public id wording. Extend test_517 coverage and regenerate its model for strict UUID validation, projection guards, and hidden @uuid_id leak prevention.
Add hash-backed UUID transaction state and a committed RT RAM @uuid_id lookup map for hot explicit UUID insert/replace paths. Keep disk/flushed chunk behavior on the existing query/SI-capable fallback path, hide the internal @uuid_id secondary index from user-facing listings, and update the UUID primary-id spec with the RAM lookup and benchmark results. Verified with: - cmake --build build --target searchd indexer indextool -j8 - php ubertest.php gen/t test_517 - php ubertest.php gen/t test_518 - php ubertest.php gen/t test_519
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add UUID as a document id type backed by an internal numeric DocID_t surrogate and hidden UUID string storage.
Related issue #1429