Skip to content

feat!: support plan extensions - #70

Merged
yongchul merged 1 commit into
mainfrom
feat/plan-extensions
Oct 7, 2026
Merged

yongchul merged 1 commit into
mainfrom
feat/plan-extensions

Conversation

@yongchul

@yongchul yongchul commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Thank you for submitting a pull request to the Substrait project. Please keep
your description clear and concise for reviewers.

Change Summary [REQUIRED]

Support plan-level advanced extensions and expected type URLs through immutable
metadata, round-trip conversion, and focused editing methods on PlanBuilder.

Motivation [REQUIRED]

Fixes #69.

Plan conversion currently drops these fields. Callers also need to set and edit
extensions during composition without constructing immutable metadata upfront
or changing previously built plans.

Reviewer Context [OPTIONAL]

  • Reuses the existing generated ReadOnlyAdvancedExtension facade. Opaque
    payloads, unknown fields, absent versus empty messages, and ordered duplicate
    type URLs survive conversion without interpretation.
  • Builder edits replace immutable metadata snapshots. Earlier plans remain
    unchanged, and repeated builds reuse unchanged metadata. Failed edits leave
    the builder's metadata intact.
  • Payload editing accepts ReadOnlyAny or IMessage: ordinary messages are
    packed immediately, while existing Any messages are copied without repacking.
    Whole-message replacement supports both read-only and protobuf extensions.
  • Removing the last payload leaves a present empty message. Only
    ClearAdvancedExtensions() removes that message; expected type URLs are edited
    independently and are never inferred from payloads.
  • All plan import entry points preserve metadata. Protobuf JSON still requires
    payload descriptors. Simple function/type extension declarations are unchanged.

Validation [REQUIRED]

  • dotnet format Substrait.sln --verify-no-changes --no-restore — Passed.
  • dotnet build Substrait.sln --configuration Release --no-restore — Passed on all targets, including netstandard2.0.
  • dotnet test --solution Substrait.sln --configuration Release --no-build — Passed: 3,888 tests, zero failures or skips, covering .NET 8 and .NET 10.

Public API and compatibility [REQUIRED]

Adds PlanMetadata, IPlan.Metadata, metadata-first plan construction, and
builder methods for setting, adding, replacing, removing, and clearing payloads
and expected type URLs. PlanBuilder.Metadata exposes an immutable snapshot for
inspection. Existing concrete plan and builder constructors remain available,
including new PlanBuilder(null).

Plan equality and hashing include metadata. No specification, dependency, or
generated-facade changes are required. The public API baseline and documentation
include the additions and migration guidance.

Breaking changes [REQUIRED]

BREAKING CHANGE: Custom IPlan implementations must provide a non-null
PlanMetadata Metadata property and recompile. Implementations without plan-level
extensions can return PlanMetadata.Empty. Existing concrete Plan constructors
default to empty metadata; serializers explicitly reject null metadata.

Summary by CodeRabbit

  • New Features
    • Added plan-level metadata for advanced extensions and ordered expected type URLs.
    • Plans can now be created with metadata, and builders can edit extensions, enhancements, optimizations, and type URLs while keeping previously built plans unchanged.
    • Plan metadata is preserved when importing and exporting plans, including opaque extension data.
  • Documentation
    • Expanded guides with metadata usage, payload handling, compatibility notes, and migration guidance for custom plan implementations.

@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a731bd22-6624-4b43-b4ff-34812637418f
📥 Commits

Reviewing files that changed from the base of the PR and between 03ac695 and c4407f6.

📒 Files selected for processing (11)
  • README.md
  • docs/metadata-facades.md
  • docs/preview-package.md
  • src/Substrait/Core/Plan/Converters/PlanToProtoConverter.cs
  • src/Substrait/Core/Plan/Converters/ProtoToPlanConverter.cs
  • src/Substrait/Core/Plan/IPlan.cs
  • src/Substrait/Core/Plan/Plan.cs
  • src/Substrait/Core/Plan/PlanBuilder.cs
  • src/Substrait/PublicAPI.Unshipped.txt
  • tests/Substrait.Tests/Core/PlanBuilderTests.cs
  • tests/Substrait.Tests/Core/RelationAnchorTests.cs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The PR adds plan-level metadata for advanced extensions and expected type URLs. Plans expose this metadata, PlanBuilder can edit it, and protobuf converters preserve it during import and export.

Changes

Plan extension metadata

Layer / File(s) Summary
Define plan metadata and expose it through plans
src/Substrait/Core/Metadata/PlanMetadata.cs, src/Substrait/Core/Plan/IPlan.cs, src/Substrait/Core/Plan/Plan.cs, src/Substrait/PublicAPI.Unshipped.txt, tests/Substrait.Tests/Core/PlanBuilderTests.cs, tests/Substrait.Tests/Core/RelationAnchorTests.cs, README.md, docs/metadata-facades.md, docs/preview-package.md
Adds immutable PlanMetadata and exposes it through IPlan and Plan. Plan construction defaults to empty metadata, and plan equality and hashing include metadata. Documentation describes the metadata contract and custom IPlan requirements.
Edit and snapshot metadata with PlanBuilder
src/Substrait/Core/Plan/PlanBuilder.cs, src/Substrait/PublicAPI.Unshipped.txt, tests/Substrait.Tests/Core/PlanBuilderMetadataTests.cs
Adds builder operations to set, clear, append, replace, and remove advanced extensions, enhancements, optimizations, and expected type URLs. Tests cover snapshots, preservation of unrelated fields, and failure behavior.
Preserve metadata in protobuf conversion
src/Substrait/Core/Plan/Converters/*, tests/Substrait.Tests/Core/PlanMetadataTests.cs, docs/metadata-facades.md
Import captures advanced extensions and expected type URLs in PlanMetadata. Export writes both to the protobuf plan and rejects plans with null metadata. Tests cover metadata round trips and payload preservation.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant ProtobufPlan
  participant ProtoToPlanConverter
  participant Plan
  participant PlanToProtoConverter
  participant SerializedPlan
  ProtobufPlan->>ProtoToPlanConverter: advanced extensions and expected type URLs
  ProtoToPlanConverter->>Plan: construct with captured PlanMetadata
  Plan->>PlanToProtoConverter: provide plan metadata
  PlanToProtoConverter->>SerializedPlan: write advanced extensions and expected type URLs
Loading

Suggested reviewers: jcamachor

Merge Risk: ⚪ Minimal · up to c4407

Plan-level extensions and expected type URLs are preserved through editing and protobuf conversion, with no actionable regression identified. The change is ready to merge after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 03ac6

The reviewed paths preserve extension data without interpreting or executing it, and metadata edits protect previously built plans. Remaining risk is concentrated in compatibility with custom plan implementations and how downstream consumers handle newly preserved payloads.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — Caller-controlled extension bytes and URL strings now survive import and can reach downstream serialized-plan consumers. The demonstrated scope is library objects and exported plans; downstream tenant isolation, execution privileges, and service exposure are unknown.

Security Findings and Attack Paths

  • observed — The reviewed conversion path forwards opaque metadata without unpacking payloads, resolving expected URLs, dynamically loading code, or executing extensions. Import captures metadata separately from ordinary extension resolution. This does not establish how external consumers interpret the exported fields.

Trust Boundaries and Controls

  • observed — Caller-owned protobuf extension messages are cloned on entry, and exported extension messages are cloned on exit. Together with immutable URL snapshots, these controls prevent ordinary subsequent caller mutation from changing stored plan metadata.

Resilience and Maintainability Implications

  • inferred — Snapshot publication contains ordinary edit failures, but builders are not thread-safe. A URL enumerable that reenters the same builder can have its intervening edit overwritten by the outer replacement. No evidence establishes a security boundary relying on concurrent or reentrant shared-builder use.
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The incremental changes add custom relation support through ExtensionLeaf, ExtensionSingle, and ExtensionMulti, schema resolution, and relation conversion. The added sections in `docs/metadata-f… Remove the custom extension-relation and schema-resolution changes from this PR, or move them to a separate PR. Keep changes needed to preserve plan-level advanced extensions and expected type URLs.
Docstring Coverage ⚠️ Warning Docstring coverage is 54.10% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 61 functions across 11 files. (4 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and identifies the main change: support for plan extensions.
Description check ✅ Passed The description covers the required change summary, motivation, validation, public API and compatibility impact, and breaking changes. Reviewer context adds useful design details.
Linked Issues check ✅ Passed Issue #69 requires plan extensions to survive import, remain accessible, and serialize. PlanMetadata and IPlan.Metadata expose advanced extensions and expected type URLs. ProtoToPlanConverter ca…
Full details: Out of Scope Changes check

Explanation

The incremental changes add custom relation support through ExtensionLeaf, ExtensionSingle, and ExtensionMulti, schema resolution, and relation conversion. The added sections in docs/metadata-facades.md and docs/preview-package.md document that feature. These changes concern custom relation operators and their output schemas, not the plan-level extensions required by issue #69.

Full details: Docstring Coverage

Explanation

Docstring coverage is 54.10% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 61 functions across 11 files. (4 skipped: 4 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 LanguageTool

LanguageTool checks are incomplete after three consecutive availability failures. Remaining chunks and files were skipped; findings from completed checks are retained.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Preserve opaque plan-level advanced extensions and expected type URLs, with immutable snapshots and focused builder editing methods.

BREAKING CHANGE: Custom IPlan implementations must provide a non-null PlanMetadata Metadata property and recompile. Implementations without plan extensions can return PlanMetadata.Empty.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@yongchul
yongchul force-pushed the feat/plan-extensions branch from 03ac695 to c4407f6 Compare October 7, 2026 03:47
@yongchul
yongchul merged commit 9275167 into main Oct 7, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Support plan extension

2 participants