The monitor resubmitted one work item only but Helix was giving 500s at that time.
The Helix submission kept retrying for 50 minutes until the job timed out.
Not sure if JobSender problem or JobMonitor one.
2026-08-11T09:30:46.5323531Z info: Monitoring Helix jobs for stage Test of build 1547565:
2026-08-11T09:30:46.5324214Z https://dev.azure.com/dnceng-public/public/_build/results?buildId=1547565
2026-08-11T09:30:46.5325090Z info: 📖 Read more about what this monitor job does: https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/Readme.md#helix-job-monitor-for-azure-devops
2026-08-11T09:30:46.7351534Z info: 🔁 Checking for failed Helix jobs to resubmit the failed work items...
2026-08-11T09:30:47.1892664Z info: 🔁 Resubmitting 1 work item(s) for job 'Windows_NT Build_Release - windows.11.amd64.client.open (1c3139e9-8f0c-41a4-818f-1b0c0ebf4a72)' (1 by Helix exit code, 0 by failed AzDO tests):
2026-08-11T09:30:47.1894050Z - Microsoft.DotNet.Helix.Sdk.Tests.dll (Helix exit code -1)
2026-08-11T09:30:47.1922038Z info: Resubmitting 1 failed work item(s) for job Windows_NT Build_Release - windows.11.amd64.client.open (1c3139e9-8f0c-41a4-818f-1b0c0ebf4a72):
2026-08-11T09:30:47.1922913Z - Microsoft.DotNet.Helix.Sdk.Tests.dll
2026-08-11T10:27:31.0026573Z ##[error]Helix Job Monitor terminated with an unhandled exception.
2026-08-11T10:27:31.0032478Z Microsoft.DotNet.Helix.Client.RestApiException`1[Microsoft.DotNet.Helix.Client.Models.ApiError]: The response contained an invalid status code 500 Internal Server Error
2026-08-11T10:27:31.0032717Z
2026-08-11T10:27:31.0033040Z Body: {"Message":"An error occured.","ActivityId":"9318133c8437fc9b11d115123aacecf1"}
2026-08-11T10:27:31.0033424Z at Microsoft.DotNet.Helix.JobMonitor.HelixService.RetryAsync[T](Func`1 action, CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/Services/HelixService.cs:line 431
2026-08-11T10:27:31.0033832Z at Microsoft.DotNet.Helix.JobMonitor.HelixService.ResubmitWorkItemsAsync(HelixJobInfo originalJob, IReadOnlyCollection`1 failedWorkItems, String targetStageAttempt, CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/Services/HelixService.cs:line 342
2026-08-11T10:27:31.0034247Z at Microsoft.DotNet.Helix.JobMonitor.JobMonitorRunner.ExecuteRetryPassAsync(CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/JobMonitorRunner.cs:line 204
2026-08-11T10:27:31.0034582Z at Microsoft.DotNet.Helix.JobMonitor.JobMonitorRunner.RunAsync(CancellationToken cancellationToken) in /_/src/Microsoft.DotNet.Helix/JobMonitor/JobMonitorRunner.cs:line 89
2026-08-11T10:27:31.0034877Z at Microsoft.DotNet.Helix.JobMonitor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.Helix/JobMonitor/Program.cs:line 86
2026-08-11T10:27:31.0163069Z
2026-08-11T10:27:31.0175554Z ##[error]Bash exited with code '1'.
2026-08-11T10:27:31.0229902Z ##[section]Finishing: Monitor Helix Jobs
The monitor resubmitted one work item only but Helix was giving 500s at that time.
The Helix submission kept retrying for 50 minutes until the job timed out.
Not sure if
JobSenderproblem orJobMonitorone.