The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2862632	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1233710	dotnet/aspnetcore	Run E2E tests	Log	dotnet/aspnetcore#64742
1233658	dotnet/maui	Controls CollectionView	Log	dotnet/maui#33177
2862466	dotnet-runtime	Run performance job script	Log
2862425	dotnet-dotnet-docker	🛡 1ES PT: AntiMalware MDE Linux Scanner (Binary) (1ES PT)	Log
1233488	dotnet/runtime	Send to Helix	Log
1233535	dotnet/roslyn	Build with analyzers	Log	dotnet/roslyn#81685
2861765	dotnet-dotnet	Build (centOSStreamContainer)	Log
1233381	dotnet/maui	Controls CollectionView	Log	dotnet/maui#33127
1233292	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#51934
2862282	dotnet-performance	Send job to Helix (Windows)	Log
2862274	dotnet-performance	Send job to Helix (Windows)	Log
2862246	dotnet-runtime	Send tests to Helix (Unix)	Log
2862266	dotnet-performance	Send job to Helix (Windows)	Log
2862254	dotnet-performance	Send job to Helix (Unix)	Log
2862239	dotnet-performance	Send job to Helix (Unix)	Log
2862223	dotnet-performance	Send job to Helix (Unix)	Log
1233092	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33183
2862174	dotnet-dotnet	Build (centOSStreamContainer)	Log
2862195	dotnet-performance	Send job to Helix (Windows)	Log
2862178	dotnet-performance	Send job to Helix (Unix)	Log
2862107	dotnet-dotnet	Prepare BuildLogs staging directory	Log
1232841	dotnet/runtime	Send to Helix	Log
2862100	dotnet-performance	Send job to Helix (Unix)	Log
2862092	dotnet-performance	Send job to Helix (Windows)	Log
1232836	dotnet/maui	Controls ScrollView	Log	dotnet/maui#33173
1231402	dotnet/maui	Controls CollectionView	Log	dotnet/maui#32674
1232765	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33161
2861948	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1232697	dotnet/maui	Controls Layout	Log	dotnet/maui#33157
1232261	dotnet/runtime	Send to Helix	Log
2861814	dotnet-runtime	Initialize containers	Log
2861809	dotnet-performance	Send job to Helix (Unix)	Log
2861773	dotnet-runtime	Build product	Log
2861757	dotnet-runtime	Send job to Helix (Windows)	Log
2861789	dotnet-performance	Send job to Helix (Unix)	Log
2861739	dotnet-runtime	Add Properties To Pipeline Env	Log
2861740	dotnet-performance	Component Detection	Log
2861730	dotnet-performance	Send job to Helix (Unix)	Log
2861731	dotnet-runtime	Build product	Log
1232258	dotnet/maui	Controls Shell	Log	dotnet/maui#33168
2861719	dotnet-performance	Send job to Helix (Windows)	Log
2861714	dotnet-performance	Component Detection	Log
2861705	dotnet-performance	Component Detection	Log
2861678	dotnet-performance	Send job to Helix (Unix)	Log
2861668	dotnet-performance	Send job to Helix (Unix)	Log
1231816	dotnet/maui	DeviceTestsMacCatalyst (Unix)	Log	dotnet/maui#33004
2861605	dotnet-performance	Component Detection	Log
2861596	dotnet-performance	Send job to Helix (Windows)	Log
2859760	dotnet-source-build-reference-packages	Validate Hosted Pool Information (1ES PT)	Log
2861469	dotnet-dotnet	Build	Log
1228398	dotnet/maui	Run Controls.Xaml.UnitTests - macOS	Log	dotnet/maui#33028
1231815	dotnet/maui	Controls CollectionView	Log	dotnet/maui#33004
2861207	dotnet-runtime	Send job to Helix (Unix)	Log
1231375	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#81693
2861112	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
2860991	dotnet-razor	🛡 Guardian: Security Analysis Report	Log
1231163	dotnet/runtime	Send tests to Helix (Unix)	Log
2860876	dotnet-runtime	Send tests to Helix (Unix)	Log
2860669	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1230918	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33152
2860233	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1230214	dotnet/runtime	Send to Helix	Log	dotnet/runtime#121412
1230448	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33144
1230240	dotnet/runtime	Send to Helix	Log
1230265	dotnet/runtime	Build native test components	Log	dotnet/runtime#122478
2860084	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
1230272	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122509
2859978	dotnet-dotnet	Build (centOSStreamContainer)	Log
1230190	dotnet/runtime	Send to Helix	Log
2859900	dotnet-workload-versions	Download secrets: EngKeyVault	Log
2859790	dotnet-dotnet	🛡 Publish SDL Results to Advanced Security (1ES PT)	Log
2859783	dotnet-dotnet	Build (fedoraContainer)	Log
1229783	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#122495
1229772	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#122497
2859767	dotnet-dotnet	Generate delegation SAS Token for dotnetbuilds/internal	Log
2859766	dotnet-arcade	Publish packages, blobs and symbols	Log
1229823	dotnet/maui	Checkout dotnet/maui@1es-devicetests to s	Log
1229699	dotnet/runtime	Build product	Log	dotnet/runtime#121305
1229650	dotnet/maui	Controls CollectionView	Log	dotnet/maui#32479
1229497	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#122388
1229598	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33134
1229583	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33132
2859634	dotnet-dotnet	Build (centOSStreamContainer)	Log
1229544	dotnet/xharness	Build and run tests	Log	dotnet/xharness#1525
2859638	dotnet-arcade-services	Publish Using Darc	Log
1228385	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33122
1229389	dotnet/runtime	Zip Build Assets	Log
1229441	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33100
1228371	dotnet/maui	Run Integration Tests - Build macOS	Log	dotnet/maui#33120
1228245	dotnet/xharness	Build and run tests	Log	dotnet/xharness#1525
1229353	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33128
1229270	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122480
1229334	dotnet/runtime	Build product	Log	dotnet/runtime#122407
2859509	dotnet-docker-tools	🟪 Generate EOL Data for all images in msdotnetcontainersstaging.azurecr.io	Log
1229167	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#81623
2859411	dotnet-runtime	Send job to Helix (Unix)	Log
1229214	dotnet/runtime	Build product	Log	dotnet/runtime#122478
1229093	dotnet/maui	Controls CollectionView	Log	dotnet/maui#33127
1228971	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122469
Displaying 100 of 1111 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
25	191	1111

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez