Skip to content

fix(groq): record only the parameters the caller passed - #3782

Open
feiiiiii5 wants to merge 2 commits into
Arize-ai:mainfrom
feiiiiii5:feiiiiii5/groq-omit-sentinel-filter
Open

feiiiiii5 wants to merge 2 commits into
Arize-ai:mainfrom
feiiiiii5:feiiiiii5/groq-omit-sentinel-filter

Conversation

@feiiiiii5

@feiiiiii5 feiiiiii5 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Description

The groq instrumentor records parameters that the caller never passed. With groq==1.0.0, this call:

client.chat.completions.create(
    messages=[{"role": "user", "content": "hi"}],
    model="llama-3.1-8b-instant",
)

records 33 entries in llm.invocation_parameters (1 expected) and 36 in input.value (2 expected):

{"model": "llama-3.1-8b-instant",
 "citation_options": "<groq.Omit object at 0x102ee8390>",
 "frequency_penalty": "<groq.Omit object at 0x102ee8390>",
 "max_tokens": "<groq.Omit object at 0x102ee8390>",
 "response_format": "<groq.Omit object at 0x102ee8390>",
 "seed": "<groq.Omit object at 0x102ee8390>",
 "stop": "<groq.Omit object at 0x102ee8390>",
 "temperature": "<groq.Omit object at 0x102ee8390>",
 "timeout": "NOT_GIVEN",
 "...": "24 more"}

So the trace claims the caller set temperature, max_tokens, top_p, seed, stop, tools, response_format, stream, … when they passed nothing, and 33 of those values are Python object addresses, so the same call records different text in every process.

Why it happens. _parse_args() binds the call, applies signature defaults, then relies on one identity test to drop the SDK's "not given" markers:

bound_signature.apply_defaults()
bound_arguments = bound_signature.arguments  # Defaults empty to NOT_GIVEN
...
if value is not None and value is not NOT_GIVEN:

That was true when the code was written and still holds on the version CI pins (test-requirements.txtgroq==0.9.0, where the optional defaults are the exported NOT_GIVEN singleton). The current SDK uses two other shapes, and neither passes the test:

parameter default type is groq.NOT_GIVEN isinstance of
temperature, tools, stream, … (33) Omit False groq.Omit
timeout NotGiven False (not the exported singleton) groq.NotGiven

pyproject.toml declares "groq >= 0.9.0" with no upper bound, so the 1.x line is supported and ci-groq-latest installs it. This is the case python/AGENTS.md names directly: "Request parameters: filter out the SDK's 'not given' sentinels (e.g. Omit / NotGiven in Stainless-generated SDKs) so unset defaults don't pollute input.value and llm.invocation_parameters."

The change. Bind without applying defaults, so an unsupplied parameter never enters the recorded request — the approach cohere/_wrappers.py already documents for the same reason ("Map the call's arguments to parameter names, keeping only values the caller set"). The existing None / NOT_GIVEN guards stay, so a caller who passes a sentinel explicitly is still filtered. No sentinel class is imported, so nothing here depends on which marker a given groq release uses — which also matters because groq < 1.0 does not export Omit publicly at all.

Only the values of two existing attributes change, and only for groq>=1.0; no attribute key is added, renamed or removed, and no span name or hierarchy is touched. That is why there is no trace screenshot below: the before/after recorded values above are the whole observable difference.

Verification

Python 3.11, one venv with the three workspace packages installed editable (openinference-semantic-conventions, openinference-instrumentation, openinference-instrumentation-groq) plus opentelemetry-sdk, wrapt, responses, httpx<0.28. Commands run from python/instrumentation/openinference-instrumentation-groq.

$ python -m pytest tests/ -q
installed groq origin/main (b737360) this branch (25bb7ea)
1.0.0 (current; ci-groq-latest) 5 failed, 16 passed 21 passed
0.9.0 (CI pin; ci-groq) 21 passed 21 passed

The 5 added tests (tests/test_invocation_parameters.py) fail on main under groq>=1.0 for the reason described above, and the existing 16 keep passing before and after. Stated plainly: under the pinned groq==0.9.0 the new tests pass on main too, because that version's defaults really are the NOT_GIVEN singleton — they bite in the groq-latest matrix environment, which python-CI.yaml does run for this package.

What they cover: omitted parameters absent from llm.invocation_parameters; absent from input.value; no <... object at 0x...> in either recorded value; temperature=0.5, max_tokens=10, stop="END" still recorded (the fix does not over-filter); and the same result on the async path.

$ ruff==0.9.2  ruff format --diff src tests   → 12 files already formatted
$ ruff==0.9.2  ruff check --no-fix src tests  → All checks passed!
$ mypy==1.11.2 mypy .                         → Success: no issues found in 12 source files

Not verified here: no live groq call (no API key), no UI rendering, and no whole-repo tox run-parallel — the diff is confined to this package, so other packages' suites were not run.

Resolves #3781

Checklist:

  • Follows OpenInference configuration to hide sensitive info — the diff only removes recorded data; it adds no field and no new value.
  • Spans properly inherit from context attributes — untouched by this diff; the context-attribute and using_attributes tests in tests/test_instrumentor.py pass on this branch.
  • Properly respects suppress tracing context — untouched; the suppress path in _start_as_current_span is not part of this diff and its tests pass.

Prepared with an AI coding assistant; the reproduction, tests and every command above were executed and inspected locally before submitting.

Parsing bound the call and then applied signature defaults, so every optional
parameter the caller left out reached the recorded request. groq >= 1.0 marks
those with an Omit instance (and timeout with a non-singleton NotGiven), which
the exported NOT_GIVEN identity test does not catch, so a two-argument
create() emitted 34 invocation parameters whose values were object addresses.

Bind without applying defaults, the way the cohere instrumentor already does,
and add regression tests for the omitted, the passed, and the async path.
AttributeValue is a union, so json.loads needs the same cast(str, ...) the
other instrumentor tests use before parsing a recorded attribute.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[bug] groq instrumentor records up to 34 parameters the caller never passed (SDK Omit defaults leak into input.value and llm.invocation_parameters)

1 participant