Skip to content

Studio: opt-in full-text analytics controls and metric view #2049

Description

@kylebernhardy

Problem

Harper's upcoming full-text analytics integration adds an opt-in fulltext metric and dynamic analytics.fullText configuration. Studio needs a discoverable way to enable and interpret it after a compatible Harper-pro release reaches Fabric.

Studio: full-text search in the schema editor, plus a search tester covers schemas, searches and readiness, not telemetry. Studio: settings forms, with Fabric-managed settings marked read-only owns generic forms. Reuse that work rather than creating another settings framework. Today's analytics settings schema rejects unknown analytics properties and does not list fullText.

Scope

  • Add capability-aware discovery and a full-text analytics view using the existing protected get_analytics operation with metric: "fulltext", instance command transport and Status analytics infrastructure. Preserve existing authorization and unsupported/error states.
  • Extend the settings schema and expose or link the relevant controls through the existing settings UI. Omit the block by default; do not silently enable collection when opening a dashboard.
analytics.fullText key Type Implied value / validation
enabled boolean false
intervalMilliseconds integer 60000; inclusive range 1000–2147483647
histograms boolean false
snapshotBytes boolean false

The core policy applies dynamically without reopening indexes. Operator collection and persistence also require analytics enabled on a writable node; analytics.aggregatePeriod: -1 or read-only mode disables operator demand. Explain these prerequisites and show disabled, unsupported and unavailable states separately. Coordinate any existing Studio save/restart behavior with the settings work; do not invent a native reopen requirement.

Interpretation and acceptance

  • Keep node/worker, index path, role method and series distinct. Use each row's sampledAt; rollups retain the latest observation per worker/index/role instead of averaging cumulative counters.
  • Preserve exact decimal-string counters and nanoseconds/bytes. Compute differences with BigInt, check safe numeric conversion for plotting, and establish a new baseline after reset, unavailable/incomplete coverage or counter decrease. Gaps are not zero activity.
  • Show availability, observation age, pending reader-byte measurements, incomplete coverage and worker-wide dropped-row counts. Distinguish cumulative counters from current queue/dispatched gauges; do not sum overlapping queue capacities or repeat worker-wide counts across indexes.
  • Optional histogram deltas cover only recording-enabled intervals. Do not invent percentiles or process/per-index resident-memory measurements from these snapshots.
  • Do not collect or introduce query text, terms, record IDs, users, raw cursors, filesystem paths or exception-message labels. Reuse the metric contract; add no new endpoint or public SDK for internal telemetry-consumer hooks.
  • Test opt-in defaults and dynamic changes, capability/permission failures, multiple workers and roles, exact counters above the safe-number range, resets, gaps, pending/error states and absent samples. Bound polling to the selected view and reuse existing query caching rather than adding parallel polls.

Dependencies and scheduling

Depends on the upcoming Harper core implementation PR and a compatible Harper-pro release deployed to Fabric. Add the core PR and actual first compatible release links when available; no release date or version is selected here. The wrapper contract is described in fulltext 0.9 observability.

This is a scoped Studio feature, not Fabric runtime implementation or generic settings-form work. No customer incident or security defect is alleged.

Comment generated by kAIle (GPT-6)

Integration PRs

Harper runtime: Add opt-in native full-text analytics #3946. Reference: Document opt-in full-text analytics and the 0.9 engine #789. Both remain draft; the compatible Harper-pro release is not yet established. Use the released public wrapper APIs through Harper; Studio should not call native handles directly.

Comment generated by kAIle (GPT-6)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions