Skip to content

feat(instrumentation): generate collector routing - #619

Merged
rapids-bot[bot] merged 4 commits into
rapidsai:mainfrom
johanpel:schema-generated-collector-integration
Sep 7, 2026
Merged

rapids-bot[bot] merged 4 commits into
rapidsai:mainfrom
johanpel:schema-generated-collector-integration

Conversation

@johanpel

@johanpel johanpel commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add schema-generated routing from collector entity stream names to typed instrumentation observers.

  • Introduce CollectorRouter and implement CollectorSink for generated model contexts.
  • Generate event deserialization and dispatch for each schema entity.
  • Expose collector wire-format serialization helpers.
  • Enable quent-events/serde directly in the collector client.

Testing

  • pixi run cargo fmt --all -- --check
  • pixi run cargo test -p quent-instrumentation-build
  • pixi run cargo check -p quent-instrumentation --features io-collector
  • pixi run cargo test -p quent-instrumentation --features io-collector --test collector_roundtrip
  • pixi run cargo test -p quent-collector-client
  • pixi run cargo test -p quent-collector -p quent-io-collector

Written by Codex.

@johanpel
johanpel force-pushed the schema-generated-collector-integration branch from 6eaca4e to f5e948a Compare August 26, 2026 12:03
@johanpel
johanpel marked this pull request as ready for review August 26, 2026 12:34
@coderabbitai

coderabbitai Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 638d9b4f-381d-496d-b643-3cee3ad8406b

📥 Commits

Reviewing files that changed from the base of the PR and between f5e948a and 7495203.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock, !Cargo.lock
📒 Files selected for processing (6)
  • Cargo.toml
  • crates/collector/client/Cargo.toml
  • crates/instrumentation-build/src/runtime/context.rs
  • crates/instrumentation/Cargo.toml
  • crates/instrumentation/src/collector.rs
  • crates/instrumentation/src/entity.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

The change adds shared collector event serialization and feature-gated collector sink support. Build options enable generated routing, which deserializes events by entity name and forwards them to matching observers.

Changes

Collector sink integration

Layer / File(s) Summary
Event serialization API
crates/collector/client/Cargo.toml, crates/collector/client/src/lib.rs, crates/collector/server/src/lib.rs
The collector client exposes serialize_event and uses it for normal processing and shutdown draining. The server re-exports the helper.
Generated collector routing
crates/instrumentation-build/src/lib.rs, crates/instrumentation-build/src/runtime/context.rs, crates/instrumentation-build/src/runtime/mod.rs
Build options enable collector sinks, reject configurations without serde, and pass the setting into generated models. Generated routing deserializes events, forwards recognized entities, and rejects unknown streams.
Instrumentation collector integration
Cargo.toml, crates/instrumentation/Cargo.toml, crates/instrumentation/src/collector.rs, crates/instrumentation/src/entity.rs, crates/instrumentation/src/lib.rs
The workspace and instrumentation manifests add the collector client. The instrumentation crate adds collector routing and sink APIs, forwards events to observers, and exposes feature-gated collector exports.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 74952

The collector integration adds typed, feature-gated event routing with roundtrip coverage, and no actionable merge-blocking risk is currently evidenced.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 8 files. (3 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: generated collector routing for instrumentation.
Description check ✅ Passed The description explains the implementation and includes testing details. It omits the Related Issues and Screenshots sections, but these omissions are non-critical for this non-UI change.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 15 functions across 8 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as spam.

}

/// Encode an [`Event`] using the collector wire format.
pub fn serialize_event<T>(event: &Event<T>) -> Result<Vec<u8>, bitcode::Error>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this could be an Event<T> method.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping this since serialization is a concern of and varies between exporters

Comment thread crates/instrumentation/src/collector.rs Outdated

/// Forwards a collected event through its entity observer.
#[doc(hidden)]
pub fn forward<E>(observer: &Observer<E>, event: crate::Event<E::Event>)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could be method of Observer<E>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread crates/instrumentation/src/entity.rs Outdated
/// Provides handles for an entity type through its shared event observer.
pub struct Observer<E: InstrumentedEntity> {
inner: Arc<ObserverInner<E::Event>>,
pub(crate) inner: Arc<ObserverInner<E::Event>>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion above avoid this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johanpel

johanpel commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot
rapids-bot Bot merged commit cd9e01d into rapidsai:main Sep 7, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants