feat(otel): add no-arg InvocationOtelPlugin constructor with ADOT Java agent SPI

[zhongkechen]

#541

Summary

• Add new InvocationOtelPlugin() for ADOT Java agent usage without handler-side GlobalOpenTelemetry initialization.
• Register OtelPluginAutoConfigurationCustomizerProvider through the OpenTelemetry Java agent autoconfigure SPI so the agent-created SdkTracerProviderBuilder gets the durable deterministic IdGenerator before the provider is built.
• Make InvocationOtelPlugin() validate that the SPI ran, then use GlobalOpenTelemetry.getTracerProvider() directly instead of creating, copying, mutating, or reflecting into a provider.
• Add MDC enrichment with trace_id/span_id on the executor thread for log-trace correlation with CloudWatch Application Signals.
• Switch examples log format from EcsLayout to Log4j2 JsonLayout for proper X-Ray log correlation via requestId.
• Move the deployed X-Ray Step/Wait examples off the old ADOT collector assumption and onto the same Java-agent extension setup as the default-constructor example.
• Package the OTel plugin jar into the examples artifact under lib/ and generate SAM config that points OTEL_JAVAAGENT_EXTENSIONS / otel.javaagent.extensions at that jar for examples that use new InvocationOtelPlugin().
• Update otel-plugin README with ADOT Java agent setup instructions and known X-Ray limitations.

How It Works Now

• The ADOT Lambda layer starts the Java process through /opt/otel-instrument.
• The generated template configures the extension path as /var/task/lib/aws-durable-execution-sdk-java-plugin-otel-.jar for no-arg OTel examples.
• The Java agent loads the plugin jar as an extension and discovers META-INF/services/io.opentelemetry.sdk.autoconfigure.spi.AutoConfigurationCustomizerProvider.
• OtelPluginAutoConfigurationCustomizerProvider installs a shared DeterministicIdGenerator on the agent SDK builder and marks installation through a system property.
• InvocationOtelPlugin() checks that marker, uses the existing global tracer provider, and keeps a separate DeterministicIdGenerator instance only to publish durable execution state through system properties. The agent-installed generator reads that state when spans are started, which works across the app and agent classloader split.

Logs correlation

• The plugin injects trace_id into MDC at invocation start (on the executor thread) so all handler-level and step-level logs include it.
• The core SDK's DurableLogger already injects requestId and executionArn into MDC.
• Log4j2 JsonLayout with properties="true" outputs all MDC fields as top-level JSON keys.
• CloudWatch parses the JSON and X-Ray correlates logs via requestId.

Why Not Reflection

• ADOT exposes Java agent wrapper providers such as ApplicationTracerProvider14, not a stable public SdkTracerProviderBuilder surface.
• Reflection paths were brittle across OpenTelemetry versions: some wrappers do not expose unobfuscate(), and internal fields such as agentTracerProvider are not consistently visible or typed from the application classloader.
• Even when an SDK provider can be reached, replacing the ID generator after the provider is already built is not a supported contract and depends on private SDK internals.
• The SPI runs at the right lifecycle point, before the agent builds the SDK provider, and is the documented OpenTelemetry Java agent extension mechanism.

Known limitations

• X-Ray ungrouped view: Plugin spans don't nest as subsegments of the Lambda platform segment (OTLP-to-X-Ray conversion limitation). Use "Group by nodes" view.
• Workflow span: Appears as a separate root segment (expected - setNoParent() with deterministic ID for cross-invocation correlation).

Testing

• mvn clean install
• Verified on Lambda with ADOT layer: traces, spans, and log correlation all working
• Verified non-otel examples still produce correct log output with JsonLayout

[SilanHe]

We'll have to add an integration test example where we use the optimized ADOT layer so that we can ensure this is working

[SilanHe]

As follow up, can we add a constructor where we also create a tracerProvider and idGenerator? this would be the case where we're using the ocmmunity collector layer + the plugin. It allows for less code to write for customers in that case.