The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2950845	dotnet-dotnet	Drift Management Checks (1ES PT Auto injected)	Log
2950872	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2950852	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2950815	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1378245	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66262
2950765	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1378133	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#126836
2950752	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1377963	dotnet/maui	Controls Shell	Log
1378171	dotnet/maui	Controls CollectionView	Log
2950742	dotnet-aspnetcore	Run build.sh	Log	#60023
1378124	dotnet/aspnetcore	Run Quarantined Tests	Log
2950593	dotnet-efcore	Cache Binary Analysis Scan Optimization (1ES PT)	Log
1378001	dotnet/runtime	Send to Helix	Log	dotnet/runtime#125607
1377967	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#126872
1377944	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#125491
1378083	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66314
1377977	dotnet/runtime	Send to Helix	Log	dotnet/runtime#123145
1378024	dotnet/maui	Controls Layout	Log
1377906	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#115996
2949761	dotnet-maui	Controls (API 30) CollectionView	Log
1377892	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126701
1377959	dotnet/maui	DeviceTestsMacCatalyst_CoreCLR (Windows)	Log	dotnet/maui#34573
1377804	dotnet/runtime	Zip Build Assets	Log
1377873	dotnet/runtime	Build product	Log	dotnet/runtime#126566
1377833	dotnet/maui	Controls WebView	Log	dotnet/maui#34935
1377802	dotnet/runtime	AOT compile CoreCLR tests	Log
2950558	dotnet-runtime	Send job to Helix (Unix)	Log
1377665	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#126864
1377729	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66303
1377735	dotnet/roslyn	Test	Log	dotnet/roslyn#83153
1377623	dotnet/maui	Controls Shell	Log
1377497	dotnet/maui	Controls CollectionView	Log
1377472	dotnet/maui	Controls SearchBar,Shape,Slider	Log
2950484	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1377624	dotnet/runtime	Send tests to Helix (Unix)	Log
2950411	dotnet-sdk	Run AoT Tests	Log	#59580
1377337	dotnet/maui	Controls CollectionView	Log	dotnet/maui#34932
2950453	dotnet-roslyn	Test	Log
2950448	dotnet-dotnet-monitor	Build	Log
2950376	dotnet-maui	Provision Android SDK - Platform APIs	Log
2950407	dotnet-runtime	Send job to Helix (Unix)	Log
2950363	dotnet-maui	Run Integration Tests - RunOniOS_BlazorDebug	Log
1377095	dotnet/maui	Controls CollectionView	Log
2950412	dotnet-sdk	Run AoT Tests	Log	#59942
1377088	dotnet/maui	Controls Shell	Log
2950392	dotnet-roslyn	Test	Log
2950331	dotnet-runtime	Send tests to Helix (Unix)	Log
2950354	dotnet-aspnetcore	Run build.cmd helix target	Log
2950345	dotnet-aspnetcore	Run build.cmd helix target	Log
1377471	dotnet/roslyn	Run Integration Tests	Log	dotnet/roslyn#81758
1377205	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#126173
2950275	dotnet-maui	Provision Android SDK - Environment variables	Log
2950335	dotnet-sdk	Run Tests in Helix	Log	#59726
2950334	dotnet-sdk	Run Tests in Helix	Log	#59725
1376955	dotnet/maui	Controls CollectionView	Log
1377207	dotnet/runtime	Run Trimming Tests	Log	dotnet/runtime#126173
1377454	dotnet/roslyn	Test	Log	dotnet/roslyn#83153
2950283	dotnet-dotnet	Build (centOSStreamContainer)	Log
1377204	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#125439
1376779	dotnet/maui	Controls Layout	Log	dotnet/maui#34573
2950294	dotnet-aspnetcore	Pre-job: Download secrets: EngKeyVault	Log	#58157
2950278	dotnet-aspnetcore	🛡 Guardian: AntiMalware Scanner (Binary)	Log	#59029
1377127	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#126427
1377119	dotnet/runtime	Publish	Log	dotnet/runtime#125298
1377116	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#126309
1377320	dotnet/roslyn	Test	Log	dotnet/roslyn#83157
1377218	dotnet/runtime	Build product	Log
2950081	dotnet-maui	Generate SBOM manifest	Log
1376773	dotnet/maui	Controls SearchBar,Shape,Slider	Log
1377098	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66251
1377000	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#126840
1376974	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#126562
1376682	dotnet/maui	Controls Image,ImageButton,IndicatorView,InputTransparent,IsEnabled,IsVisible	Log
2950119	dotnet-dotnet	Run Tests	Log
2950114	dotnet-dotnet	Run Tests	Log
1376899	dotnet/runtime	Publish	Log	dotnet/runtime#126836
2950152	dotnet-efcore	Build	Log
2950142	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
1376876	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#115996
1377062	dotnet/arcade	Run Helix Tests	Log	dotnet/arcade#16711
2950001	dotnet-maui	Component Detection (auto-injected by policy)	Log
1376923	dotnet/aspnetcore	Run Quarantined Tests	Log
1376549	dotnet/maui	Controls Shell	Log
2950074	dotnet-wpf	🔒 Start Network Isolation	Log	#59385
2950077	dotnet-arcade	Publish packages, blobs and symbols	Log
2949998	dotnet-maui	Run Integration Tests - RunOniOS_MauiRelease	Log
2950048	dotnet-maui	Controls (vlatest) Label	Log
2950068	dotnet-wpf	Generate delegation SAS Token for dotnetbuilds/internal	Log	#59386
1376800	dotnet/runtime	Zip	Log	dotnet/runtime#125180
2950056	dotnet-arcade	Publish packages, blobs and symbols	Log
2949744	dotnet-maui	Controls Shell	Log
2950033	dotnet-dotnet	Run Tests (azurelinuxSourceBuildTestContainer)	Log
2949865	dotnet-maui	Run Integration Tests - RunOniOS_MauiNativeAOT	Log
2949934	dotnet-runtime	Send to Helix	Log
1376744	dotnet/runtime	Zip	Log	dotnet/runtime#126834
2949920	dotnet-maui	Controls Editor,Effects,Essentials,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log
1376719	dotnet/maui	Controls Editor,Effects,Essentials,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#34932
1376877	dotnet/roslyn	Test	Log	dotnet/roslyn#83153
2949884	dotnet-maui	Run Integration Tests - RunOniOS_MauiRelease	Log
Displaying 100 of 2845 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
93	616	2845

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez