Skip to content

Fix TelemetryTests flake from parallel dotnet test races#8292

Open
Evangelink wants to merge 1 commit into
mainfrom
dev/copilot/fix-telemetry-flake
Open

Fix TelemetryTests flake from parallel dotnet test races#8292
Evangelink wants to merge 1 commit into
mainfrom
dev/copilot/fix-telemetry-flake

Conversation

@Evangelink
Copy link
Copy Markdown
Member

Problem

TelemetryTests.VSTest_RunTests_Succeeds and VSTest_DiscoverTests_Succeeds are parameterised by target framework via [DynamicData(TargetFrameworks.AllForDynamicData)] and both invoke:

dotnet test -c Release {AssetFixture.VSTestProjectPath} --framework {tfm}

against the same shared vstest/TelemetryVSTestProject.csproj. With MSTest parallelism enabled, multiple TFM instances run concurrently and race on MSBuild outputs such as bin/Release/<tfm>/TelemetryVSTestProject.runtimeconfig.json and shared obj/ files.

Observed CI failure:

error MSB4018: The ""GenerateRuntimeConfigurationFiles"" task failed unexpectedly.
System.IO.IOException: The process cannot access the file
'...\TelemetryTests\vstest\bin\Release\net10.0\TelemetryVSTestProject.runtimeconfig.json'
because it is being used by another process.

Fix

Apply [DoNotParallelize] at the TelemetryTests class level to serialize all tests in the class. This is a one-line change, preserves test coverage, and only has a small impact on suite duration.

Blocked PRs

This flake has been intermittently blocking: #8284 #8259 #8234 #8268 #8270 #8256 #8255 #8189 #8262 #8237 #8231

The VSTest_RunTests_Succeeds and VSTest_DiscoverTests_Succeeds tests both invoke `dotnet test` against the same shared VSTest project path for every target framework. Running them in parallel races on MSBuild outputs such as `bin/Release/<tfm>/TelemetryVSTestProject.runtimeconfig.json` and shared `obj/` files, which causes intermittent `GenerateRuntimeConfigurationFiles` failures (file in use).

Apply `[DoNotParallelize]` at the class level to serialize all tests in TelemetryTests. Coverage is unchanged and the duration impact is small.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 16, 2026 14:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an intermittent flake in MSTest.Acceptance.IntegrationTests caused by MSTest’s assembly-level parallel execution running multiple target-framework instances of the VSTest telemetry tests concurrently, which races on shared bin/ and obj/ MSBuild outputs.

Changes:

  • Add [DoNotParallelize] at the TelemetryTests class level to prevent parallel execution of its test methods / data rows.
  • Document the underlying MSBuild output race causing GenerateRuntimeConfigurationFiles failures.
Show a summary per file
File Description
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/TelemetryTests.cs Serializes TelemetryTests execution via [DoNotParallelize] to avoid dotnet test output races across TFMs.

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 0

Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of PR #8292[DoNotParallelize] on TelemetryTests

Summary: Adds [DoNotParallelize] to TelemetryTests to prevent intermittent race conditions on shared MSBuild outputs (bin/, obj/) when dotnet test is invoked against the same VSTest project path across multiple target frameworks in parallel.

Assessment

The fix is correct and well-motivated. Shared MSBuild intermediate/output directories are a well-known source of non-deterministic failures when the same project is built concurrently, and [DoNotParallelize] is the appropriate MSTest mechanism to serialize tests within a class.

No bugs, logic errors, security issues, or API surface concerns were identified. The explanatory comment accurately describes the root cause and chosen mitigation.

No changes requested.

Generated by Expert Code Review (on open) for issue #8292 · ● 1.8M

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants