feat(rails): report Active Job retry failures

.changeset/rails-job-retry-reporting.md

@@ -0,0 +1,5 @@
+---
+"posthog-rails": minor
+---
+
+Add opt-in `report_active_job_retries` reporting for Active Job retry failures, reusing job identity, filtered arguments, and core exception deduplication. The option defaults to false and requires `auto_instrument_active_job`. Retry reports support Rails 5.2 and honor exception exclusions across the serialized cause chain without changing terminal reporting.

posthog-rails/README.md

@@ -45,6 +45,8 @@ Live reporters rebuild context from their controller's own filtered request data
 
 ## Active Job context
 
+`config.report_active_job_retries = true` additionally reports failures when Active Job schedules a retry. It defaults to `false` and requires `auto_instrument_active_job`. Retry reports use the same effective identity and filtered job properties as terminal errors; discarded or otherwise rescued failures are not added. Retry reports honor default and configured `excluded_exceptions` by exact, case-sensitive Ruby class name across the serialized cause chain (up to 50 entries, with cycles visited once). Excluded failures still schedule their retries. This option also works on Rails 5.2, which does not emit retry notifications. Existing terminal reporting is unchanged. Core exception deduplication still applies, including when the same exception object is raised again. New retry reports use level `error`, source `rails.active_job`, and mechanism `task` with `handled: true` because the Active Job retry handler consumes the failure and reschedules the job. Terminal failures that escape job execution remain unhandled and retain their existing metadata.
+
 Set `config.auto_instrument_active_job = true` to copy the current session and distinct IDs into queued Active Job data. Each job runs with its saved IDs in a fresh context. Request properties are excluded. The context is restored after success, an error, or a retry. Older jobs without saved IDs run with empty context.
 
 The existing job distinct ID extractor takes precedence over the saved ID. Rescue, discard, and terminal retry handlers keep this effective context, including when they enqueue child jobs. Integer and finite floating-point distinct IDs are stored as strings in job snapshots; session IDs must be strings. Events, optional logs, and application OpenTelemetry spans use the shared job context. Register `PostHog::ContextSpanProcessor` for child spans as described in the trace context guide. An existing job span receives IDs only when its public `messaging.system` is `active_job` and `messaging.message.id` matches the current job ID. Other active spans stay unchanged. This transfers PostHog IDs; it does not transfer a distributed trace or configure a tracing provider.

posthog-rails/examples/posthog.rb

@@ -22,6 +22,10 @@
   # Set to true to enable automatic ActiveJob exception tracking
   # config.auto_instrument_active_job = true
 
+  # Report failures when Active Job schedules a retry (default: false)
+  # Requires auto_instrument_active_job; terminal unhandled failures are already reported.
+  # config.report_active_job_retries = true
+
   # Use PostHog tracing headers for request-scoped identity/session context (default: true)
   # Request metadata (current URL, method, path, user agent, and IP) is always captured during Rails requests
   # Set to false to ignore client-supplied X-PostHog-Distinct-Id and X-PostHog-Session-Id headers

posthog-rails/lib/generators/posthog/templates/posthog.rb

@@ -22,6 +22,10 @@
   # Set to true to enable automatic ActiveJob exception tracking
   # config.auto_instrument_active_job = true
 
+  # Report failures when Active Job schedules a retry (default: false)
+  # Requires auto_instrument_active_job; terminal unhandled failures are already reported.
+  # config.report_active_job_retries = true
+
   # Use PostHog tracing headers for request-scoped identity/session context (default: true)
   # Request metadata (current URL, method, path, user agent, and IP) is always captured during Rails requests
   # Set to false to ignore client-supplied X-PostHog-Distinct-Id and X-PostHog-Session-Id headers

posthog-rails/lib/posthog/rails/active_job.rb

@@ -13,8 +13,34 @@ module ActiveJobExtensions
       def self.prepended(base)
         base.extend(ClassMethods)
         base.around_perform :with_posthog_job_context, prepend: true if base.respond_to?(:around_perform)
+        base.prepend(LegacyRetryReporting) if ::ActiveJob::VERSION::MAJOR < 6
+        return if @retry_subscription
+
+        @retry_subscription = ActiveSupport::Notifications.subscribe('enqueue_retry.active_job') do |*args|
+          payload = args.last
+          job = payload[:job]
+          job.send(:capture_posthog_retry, payload[:error]) if job.is_a?(ActiveJobExtensions)
+        end
       end
 
+      # Active Job 5.2 neither emits enqueue_retry nor passes the error to retry_job.
+      module LegacyRetryReporting
+        # @api private
+        def rescue_with_handler(exception, ...)
+          previous_exception = @posthog_retry_exception
+          @posthog_retry_exception = exception
+          super
+        ensure
+          @posthog_retry_exception = previous_exception
+        end
+
+        # @api private
+        def retry_job(...)
+          super.tap { capture_posthog_retry(@posthog_retry_exception) }
+        end
+      end
+      private_constant :LegacyRetryReporting
+
       module ClassMethods
         # DSL for defining how to extract distinct_id from job arguments.
         #
@@ -148,7 +174,34 @@ def posthog_job_distinct_id
         nil
       end
 
-      def capture_job_exception(exception)
+      def capture_posthog_retry(exception)
+        config = PostHog::Rails.config
+        return unless config&.auto_instrument_active_job && config.report_active_job_retries && exception
+        return unless PostHog.initialized? && PostHog.client.enabled?
+        return if excluded_posthog_retry?(exception, config)
+
+        Internal::Context.with_context(
+          @posthog_execution_context || posthog_job_context, fresh: true, enrich_current_span: false
+        ) do
+          capture_job_exception(exception, retry_failure: true)
+        end
+      rescue StandardError => e
+        PostHog::Logging.logger.error("Failed to capture job retry: #{e.class}")
+      end
+
+      def excluded_posthog_retry?(exception, config)
+        seen = {}.compare_by_identity
+        current = exception
+        while current && seen.size < PostHog::ExceptionCapture::MAX_CHAINED_EXCEPTIONS && !seen.key?(current)
+          return true unless config.should_capture_exception?(current)
+
+          seen[current] = true
+          current = current.respond_to?(:cause) ? current.cause : nil
+        end
+        false
+      end
+
+      def capture_job_exception(exception, retry_failure: false)
         return unless PostHog::Rails.config&.auto_instrument_active_job
 
         # Build distinct_id from job arguments if possible
@@ -159,22 +212,24 @@ def capture_job_exception(exception)
                       end
 
         properties = {
-          '$exception_source' => 'active_job',
+          '$exception_source' => retry_failure ? 'rails.active_job' : 'active_job',
           '$job_class' => self.class.name,
           '$job_id' => job_id,
           '$queue_name' => queue_name,
           '$job_priority' => priority,
           '$job_executions' => executions
         }
 
+        properties['$exception_level'] = 'error' if retry_failure
+
         # Add serialized job arguments (be careful with sensitive data)
         properties['$job_arguments'] = sanitize_job_arguments(arguments) if arguments.present?
 
         PostHog.capture_exception(
           exception,
           distinct_id,
           properties,
-          mechanism: { 'type' => 'active_job', 'handled' => false }
+          mechanism: { 'type' => retry_failure ? 'task' : 'active_job', 'handled' => retry_failure }
         )
       rescue StandardError => e
         # Don't let PostHog errors break job processing

posthog-rails/lib/posthog/rails/configuration.rb

@@ -19,6 +19,10 @@ class Configuration
       # @return [Boolean] Whether to automatically instrument ActiveJob. Defaults to false.
       attr_accessor :auto_instrument_active_job
 
+      # @return [Boolean] Whether to report each Active Job retry failure. Requires auto_instrument_active_job.
+      #   Defaults to false.
+      attr_accessor :report_active_job_retries
+
       # @return [Array<String>] Exception class names to ignore in addition to the defaults.
       attr_accessor :excluded_exceptions
 
@@ -73,6 +77,7 @@ def initialize
         @auto_capture_exceptions = false
         @report_rescued_exceptions = false
         @auto_instrument_active_job = false
+        @report_active_job_retries = false
         @excluded_exceptions = []
         @use_tracing_headers = true
         @capture_user_context = true

public_api_snapshot.txt

@@ -391,6 +391,8 @@ instance_method PostHog::Rails::Configuration#logs_max_records_per_minute()
 instance_method PostHog::Rails::Configuration#logs_max_records_per_minute=(arg0)
 instance_method PostHog::Rails::Configuration#logs_notifications()
 instance_method PostHog::Rails::Configuration#logs_notifications=(arg0)
+instance_method PostHog::Rails::Configuration#report_active_job_retries()
+instance_method PostHog::Rails::Configuration#report_active_job_retries=(arg0)
 instance_method PostHog::Rails::Configuration#report_rescued_exceptions()
 instance_method PostHog::Rails::Configuration#report_rescued_exceptions=(arg0)
 instance_method PostHog::Rails::Configuration#should_capture_exception?(exception)

spec/fixtures/active_job_retry_lifecycle.rb

@@ -0,0 +1,169 @@
+# frozen_string_literal: true
+
+require 'logger'
+require 'json'
+require 'active_job'
+require 'posthog'
+
+# Standalone Active Job harness, not a booted Rails application. These accessors
+# supply only the Rails integration's version and absent application/config host.
+module Rails
+  def self.version
+    ActiveJob::VERSION::STRING
+  end
+
+  def self.application
+    nil
+  end
+end
+
+require 'posthog/rails/configuration'
+require 'posthog/rails/active_job'
+require 'posthog/rails/facade'
+
+module PostHog
+  module Rails
+    class << self
+      attr_accessor :config
+    end
+  end
+end
+PostHog::Rails.install_posthog_facade!
+PostHog::Rails.config = PostHog::Rails::Configuration.new
+PostHog::Rails.config.auto_instrument_active_job = true
+PostHog::Logging.logger = Logger.new(File::NULL)
+PostHog.init(api_key: 'phc_fixture', test_mode: true)
+
+# Active Job 5.2's test adapter retains arguments, not the full serialized job.
+class RetryLifecycleAdapter < ActiveJob::QueueAdapters::TestAdapter
+  attr_reader :payload
+
+  def enqueue(job)
+    @payload = JSON.parse(JSON.generate(job.serialize))
+    super
+  end
+
+  def enqueue_at(job, timestamp)
+    @payload = JSON.parse(JSON.generate(job.serialize))
+    super
+  end
+end
+
+class RetryLifecycleJob < ActiveJob::Base
+  prepend PostHog::Rails::ActiveJobExtensions
+
+  self.queue_adapter = RetryLifecycleAdapter.new
+  self.logger = Logger.new(File::NULL)
+  class_attribute :handler_calls, default: 0
+  class_attribute :last_failure
+  posthog_distinct_id { 'job-person' }
+  retry_on RuntimeError, wait: 0, attempts: 2
+
+  def perform
+    self.class.last_failure = RuntimeError.new('lifecycle failure')
+    raise self.class.last_failure
+  end
+end
+
+context = PostHog.const_get(:Internal).const_get(:Context)
+client = PostHog.client
+config = PostHog::Rails.config
+
+def assert_lifecycle(condition, message)
+  raise message unless condition
+end
+
+def lifecycle_event
+  PostHog.client.dequeue_last_message if PostHog.client.queued_messages.positive?
+end
+
+def lifecycle_payload
+  RetryLifecycleJob.queue_adapter.payload
+end
+
+[false, true].each do |enabled|
+  config.report_active_job_retries = enabled
+  context.with_context(distinct_id: 'saved-person', session_id: 'saved-session',
+                       properties: { 'request' => 'secret' }) do
+    RetryLifecycleJob.perform_later
+  end
+  before_retry = RetryLifecycleJob.queue_adapter.enqueued_jobs.size
+  context.with_context(distinct_id: 'worker-person', session_id: 'worker-session') do
+    outer = context.current
+    result = ActiveJob::Base.execute(lifecycle_payload)
+    assert_lifecycle(result.equal?(RetryLifecycleJob.last_failure), 'handler return changed')
+    assert_lifecycle(context.current.equal?(outer), 'worker context changed')
+  end
+  assert_lifecycle(RetryLifecycleJob.queue_adapter.enqueued_jobs.size == before_retry + 1, 'retry not scheduled once')
+  event = lifecycle_event
+  assert_lifecycle(!event.nil? == enabled, "expected #{enabled ? 1 : 0} retry events, got #{event ? 1 : 0}")
+  if event
+    assert_lifecycle(event[:distinct_id] == 'job-person', 'extractor identity lost')
+    assert_lifecycle(event[:properties]['$session_id'] == 'saved-session', 'saved session lost')
+    assert_lifecycle(event[:properties]['$exception_source'] == 'rails.active_job', 'retry source lost')
+    mechanism = event[:properties]['$exception_list'].first['mechanism']
+    assert_lifecycle(mechanism['type'] == 'task' && mechanism['handled'] == true, 'retry not marked handled')
+    assert_lifecycle(!event[:properties].key?('request'), 'request properties leaked')
+  end
+  begin
+    ActiveJob::Base.execute(lifecycle_payload)
+    raise 'terminal error swallowed'
+  rescue RuntimeError => e
+    assert_lifecycle(e.equal?(RetryLifecycleJob.last_failure), 'terminal error replaced')
+  end
+  terminal_event = lifecycle_event
+  assert_lifecycle(terminal_event&.dig(:properties, '$exception_source') == 'active_job', 'terminal event missing')
+  mechanism = terminal_event[:properties]['$exception_list'].first['mechanism']
+  assert_lifecycle(mechanism['type'] == 'active_job' && mechanism['handled'] == false, 'terminal metadata changed')
+  assert_lifecycle(lifecycle_event.nil?, 'duplicate event')
+  assert_lifecycle(context.current.nil?, 'context leaked')
+end
+
+config.report_active_job_retries = true
+%i[discard rescue terminal_retry].each do |handler|
+  klass = Class.new(RetryLifecycleJob)
+  Object.const_set("RetryLifecycle#{handler.to_s.camelize}Job", klass)
+  case handler
+  when :discard then klass.discard_on(RuntimeError) { klass.handler_calls += 1 }
+  when :rescue then klass.rescue_from(RuntimeError) { klass.handler_calls += 1 }
+  when :terminal_retry then klass.retry_on(RuntimeError, attempts: 1) { klass.handler_calls += 1 }
+  end
+  klass.perform_now
+  assert_lifecycle(klass.handler_calls == 1, "#{handler} handler called more than once")
+  assert_lifecycle(lifecycle_event.nil?, "#{handler} gained a report")
+end
+
+config.excluded_exceptions = ['ArgumentError']
+class ExcludedRetryLifecycleJob < RetryLifecycleJob
+  def perform
+    raise ArgumentError, 'excluded'
+  rescue ArgumentError
+    raise 'wrapper'
+  end
+end
+before_retry = RetryLifecycleJob.queue_adapter.enqueued_jobs.size
+ExcludedRetryLifecycleJob.perform_now
+assert_lifecycle(RetryLifecycleJob.queue_adapter.enqueued_jobs.size == before_retry + 1, 'excluded retry not scheduled')
+assert_lifecycle(lifecycle_event.nil?, 'excluded retry reported')
+
+job = RetryLifecycleJob.new
+job.perform_now
+lifecycle_event
+result = job.retry_job(wait: 0)
+assert_lifecycle(result.is_a?(RetryLifecycleJob) && result.job_id == job.job_id, 'retry_job return changed')
+assert_lifecycle(result.equal?(job), 'legacy retry_job no longer returns self') if ActiveJob::VERSION::MAJOR < 6
+assert_lifecycle(lifecycle_event.nil?, 'stale handler error reported')
+
+config.auto_instrument_active_job = false
+RetryLifecycleJob.perform_now
+assert_lifecycle(lifecycle_event.nil?, 'disabled instrumentation reported')
+config.auto_instrument_active_job = true
+PostHog.client = nil
+RetryLifecycleJob.perform_now
+assert_lifecycle(PostHog.client.nil?, 'uninitialized client replaced')
+PostHog.init(api_key: '', test_mode: true, silence_disabled_client_error: true)
+RetryLifecycleJob.perform_now
+assert_lifecycle(lifecycle_event.nil?, 'disabled client reported')
+PostHog.client.shutdown
+client.shutdown
+puts "Active Job #{ActiveJob::VERSION::STRING}: retry lifecycle controls passed"