Default datadog.logs.autoMultiLineDetection to true - #2841
Conversation
This comment has been minimized.
This comment has been minimized.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dfb1477226
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ## ref: https://docs.datadoghq.com/agent/logs/advanced_log_collection/?tab=configurationfile#automatic-multi-line-aggregation | ||
| autoMultiLineDetection: false | ||
| ## ref: https://docs.datadoghq.com/agent/logs/auto_multiline_detection/ | ||
| autoMultiLineDetection: true |
There was a problem hiding this comment.
When users enable log collection but leave datadog.logs.autoMultiLineDetection unset in their values file, this new default makes the Helm-rendered Agent run with DD_LOGS_CONFIG_AUTO_MULTI_LINE_DETECTION=true; however the migration ConfigMap writes only datadog.operator.migration.userValues to the mapper (charts/datadog/templates/migration-values-configmap.yaml), so the unset key is omitted from spec.features.logCollection.autoMultiLineDetection. Since the DatadogAgent CRD does not default that field, those migrated installs fall back to the Operator/Agent default instead of preserving the Helm behavior. Please inject this chart default into the migration path or otherwise normalize the migration values.
Useful? React with 👍 / 👎.
272720e to
542c694
Compare
github-actions
Bot
added
the
chart/datadog
|
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 542c694e06
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ## ref: https://docs.datadoghq.com/agent/logs/advanced_log_collection/?tab=configurationfile#automatic-multi-line-aggregation | ||
| autoMultiLineDetection: false | ||
| ## ref: https://docs.datadoghq.com/agent/logs/auto_multiline_detection/ | ||
| autoMultiLineDetection: true |
There was a problem hiding this comment.
For upgrades where users left this value unset, this flips log aggregation from false to true immediately on helm upgrade, so existing installs silently opt in to the new behavior. The datadog chart review guidance treats stable default changes as upgrade-breaking unless they ship with a deprecation/migration path or a major-version boundary; a minor bump plus changelog opt-out still changes running installs that do not already pin this value. Please keep the default stable until a major release, or add an upgrade-safe compatibility path.
AGENTS.md reference: AGENTS.md:L7-L7
Useful? React with 👍 / 👎.
V2 automatic multi-line log detection becomes the default agent behavior (AGTINFR-744). Also fixes CONS-7733: DD_LOGS_CONFIG_AUTO_MULTI_LINE_DETECTION was rendered twice with conflicting values when a manual agents.containers.agent.env entry set the same variable as datadog.logs.autoMultiLineDetection; the chart-driven entry is now skipped when the user already supplies one via env.
datadog.env and datadog.envDict are rendered by containers-common-env before the chart-driven DD_LOGS_CONFIG_AUTO_MULTI_LINE_DETECTION entry, so an override via those (or agents.containers.agent.envDict) wasn't being detected, letting the chart-driven value still emit alongside it.
The helm-vs-operator config mapper only carries a values.yaml key into the generated DDA when it's explicitly set in the source file. Since the operator's own default for autoMultiLineDetection is still false, fixtures that relied on the (now-true) Helm chart default diverged from the operator-rendered config in the yamlmapper integ tests. Set the field explicitly to match, mirroring the existing apm-logs/full-observability fixtures.
…takes effect The datadog-operator's logcollection feature only applies any of its sub-settings (including autoMultiLineDetection) when spec.features.logCollection.enabled is true — otherwise Configure() short-circuits and the whole feature block is left at its zero value. The prior fixture fix set autoMultiLineDetection alone, which the mapper carried into the DDA correctly but the operator ignored since logCollection.enabled was never set. Add datadog.logs.enabled: true alongside it, matching the working apm-logs/full-observability fixtures.
Mirrors the existing Operator-enabled-by-default and registryMigrationMode notices: explains the change at helm install/upgrade time and gives the exact command to opt out.
542c694 to
2f7b48b
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
3b58cfd
into
main
2 participants
Replaces #2801 — re-opened from a branch on
DataDog/helm-chartsdirectly (instead of a fork) so CI automation (labeler, chart-version bump/validation) has write access.What does this PR do?
Flips the default of
datadog.logs.autoMultiLineDetectionfromfalsetotrue. Automatic multi-line log detection (V2) becomes the default Agent behavior, targeting Agent 7.82.x. See https://docs.datadoghq.com/agent/logs/auto_multiline_detection/Related: AGTINFR-744
Also fixes CONS-7733:
DD_LOGS_CONFIG_AUTO_MULTI_LINE_DETECTIONwas rendered twice with conflicting values whenever a manualagents.containers.agent.enventry set the same variable thatdatadog.logs.autoMultiLineDetectionalso sets. This bug becomes much more visible now thatfalseis the value users need to set to opt back out, rather thantrueto opt in. The chart-driven entry is now skipped if the user already supplies one viaenv.AGNTLOG-126 (helm chart bindings for Auto Multiline V2) is resolved Done via #1764, which only added an
autoMultiLineExtraPatternsbinding and is superseded by this PR for the actual default flip and the env-override fix above.Motivation
V2 auto multi-line detection has been validated with customers (Org2, OpenAI, US Bank) and addresses pain points with the current aggregation strategy. This PR makes it the default.
Review checklist (self-checked)
CHANGELOG.mdis updatedREADME.mdis updated withhelm-docsAdditional Notes
Also fixes 13
test/datadog/yamlmapperfixtures that relied on the (now-true) Helm default rather than settingdatadog.logs.enabled/autoMultiLineDetectionexplicitly. The Helm-vs-Operator mapper only carries values into the generated DatadogAgent CR when present in the source file, and the Operator only applies anylogCollectionsub-setting whenlogCollection.enabledis true — so these fixtures diverged from the Operator's still-falsebinary default once the Helm chart flipped its own default.Chart version bumped
3.234.0→3.235.0(minor, per this repo's convention of minor bumps for default-behavior changes affecting all existing users).🤖 Generated with Claude Code