Skip to content

Prevent telemetry host termination on boolean MSTest setting payloads#8189

Open
Copilot wants to merge 4 commits into
mainfrom
copilot/fix-test-failures-assert-equal
Open

Prevent telemetry host termination on boolean MSTest setting payloads#8189
Copilot wants to merge 4 commits into
mainfrom
copilot/fix-test-failures-assert-equal

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 13, 2026

Bug Fix

What was the bug?

Telemetry acceptance runs were failing on Linux Debug with exit code 134 because AppInsightsProvider hit a debug assertion when a telemetry payload contained a bool (mstest.setting.parallelization_enabled). That assertion terminated the host during telemetry ingestion.

How did you fix it?

  • Telemetry ingestion hardening
    • Updated AppInsightsProvider to treat bool payload values as valid input and serialize them via AsTelemetryBool() without asserting.
    • This preserves expected telemetry normalization ("true" / "false") while removing a process-killing path in debug builds.
  • Regression coverage
    • Added a focused unit test in AppInsightsProviderTests that logs a payload containing a boolean property and verifies the emitted property is TelemetryProperties.True.
case bool value:
    properties.Add(pair.Key, value.AsTelemetryBool());
    break;

Testing

  • Added/ran targeted unit coverage for boolean telemetry conversion in AppInsightsProviderTests.
Original prompt

Fix the following test failures

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs#L259

src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs(259,1): error : [MTP_DiscoverTests_SendsTelemetryEvent ("net10.0")] [net11.0] Assert.AreEqual failed. Expected:<0>. Actual:<134>. 'expected' expression: 'exitCode', 'actual' expression: 'testHostResult.ExitCode'. Expression 'AssertExitCodeIs' failed for member 'MTP_DiscoverTests_SendsTelemetryEvent' at line 64 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/TelemetryTests.cs'. Output of the test host is:
Command: /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/2vdru/TelemetryTests/bin/Release/net10.0/TelemetryMTPProject --list-tests --diagnostic

ExitCode: 134

StandardOutput:
MSTest v4.3.0-ci (UTC 05/13/2026) [linux-x64 - .NET 10.0.8]
Diagnostic file (level 'Trace' with async flush): /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/2vdru/TelemetryTests/bin/Release/net10.0/TestResults/log_260513163920798.diag
PassingTest
DataDrivenTest (1)
DataDrivenTest (2)
TestWithTimeout
Test discovery summary: found 4 test(s) - /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/2vdru/TelemetryTests/bin/Release/net10.0/TelemetryMTPProject.dll (net10.0|x64)
duration: 1s 265ms

StandardError:
Process terminated.
Assertion failed.
Telemetry entry 'mstest.setting.parallelization_enabled' contains a boolean value, boolean values should always be converted to string using: .AsTelemetryBool()
at Microsoft.Testing.Platform.RoslynDebug.Assert(Boolean b, String message) in //src/Platform/Microsoft.Testing.Platform/Helpers/RoslynDebug.cs:line 21
at Microsoft.Testing.Extensions.Telemetry.AppInsightsProvider.IngestLoopAsync() in /
/src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs:line 192
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.Testing.Extensions.Telemetry.AppInsightsProvider.IngestLoopAsync()
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()
Check failure on line 259 in src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs#L259

src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs(259,1): error : [MTP_RunTests_SendsTelemetryWithSettingsAndAttributes ("net10.0")] [net11.0] Assert.AreEqual failed. Expected:<0>. Actual:<134>. 'expected' expression: 'exitCode', 'actual' expression: 'testHostResult.ExitCode'. Expression 'AssertExitCodeIs' failed for member 'MTP_RunTests_SendsTelemetryWithSettingsAndAttributes' at line 31 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/TelemetryTests.cs'. Output of the test host is:
Command: /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/2vdru/TelemetryTests/bin/Release/net10.0/TelemetryMTPProject --diagnostic

ExitCode: 134

StandardOutput:
MSTest v4.3.0-ci (UTC 05/13/2026) [linux-x64 - .NET 10.0.8]
Diagnostic file (level 'Trace' with async flush): /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/2vdru/TelemetryTests/bin/Release/net10.0/TestResults/log_260513163920757.diag
Telemetry

Microsoft Testing Platform collects usage data in order to help us improve your experience. The data is collected by Microsoft and are not shared with anyone.
You can opt-out of telemetry by setting the TESTINGPLATFORM_TELEMETRY_OPTOUT or DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
Read more about Microsoft Testing Platform telemetry: https://aka.ms/testingplatform/telemetry

StandardError:
Process terminated.
Assertion Failed
Telemetry entry 'mstest.setting.parallelization_enabled' contains a boolean value, boolean values should always be converted to string using: .AsTelemetryBool()
at Microsoft.Testing.Platform.RoslynDebug.Assert(Boolean b, String message) in //src/Platform/Microsoft.Testing.Platform/Helpers/RoslynDebug.cs:line 21
at Microsoft.Testing.Extensions.Telemetry.AppInsightsProvider.IngestLoopAsync() in /
/src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs:line 192
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ...

Created from VS Code.

Copilot AI self-assigned this May 13, 2026
Copilot AI review requested due to automatic review settings May 13, 2026 16:49
Copilot AI review requested due to automatic review settings May 13, 2026 16:49
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot May 13, 2026 17:02
Copilot AI changed the title [WIP] Fix test failures in Assert.AreEqual for telemetry events Prevent telemetry host termination on boolean MSTest setting payloads May 13, 2026
Copilot AI requested a review from Evangelink May 13, 2026 17:04
@Evangelink Evangelink marked this pull request as ready for review May 14, 2026 12:34
Copilot AI review requested due to automatic review settings May 14, 2026 12:34
Comment thread src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs Outdated
Comment thread src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs Outdated
Co-authored-by: Amaury Levé <amauryleve@microsoft.com>
@Evangelink
Copy link
Copy Markdown
Member

/review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 14, 2026

Expert Code Review (command) failed. Please review the logs for details.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens telemetry ingestion so boolean payload values no longer terminate debug hosts, and adds regression coverage for boolean telemetry normalization.

Changes:

  • Converts boolean telemetry payload values with AsTelemetryBool().
  • Adds a unit test verifying boolean telemetry properties are emitted as telemetry strings.
Show a summary per file
File Description
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs Removes the debug assertion path for boolean telemetry values and serializes them normally.
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AppInsightsProviderTests.cs Adds regression coverage for logging a boolean telemetry property.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment on lines +200 to +201

await appInsightsProvider.DisposeAsync();
@Evangelink
Copy link
Copy Markdown
Member

@copilot address review comments and fix the following issues:

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs#L87

src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs(87,9): error CS0103: (NETCORE_ENGINEERING_TELEMETRY=Build) The name '_task' does not exist in the current context
Check failure on line 264 in src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs#L264

src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs(264,9): error CS0103: (NETCORE_ENGINEERING_TELEMETRY=Build) The name 'RoslynDebug' does not exist in the current context
Check failure on line 87 in src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs#L87

src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs(87,9): error CS0103: (NETCORE_ENGINEERING_TELEMETRY=Build) The name '_task' does not exist in the current context
Check failure on line 264 in src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs#L264

src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs(264,9): error CS0103: (NETCORE_ENGINEERING_TELEMETRY=Build) The name 'RoslynDebug' does not exist in the current context
Check failure on line 87 in src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs#L87

src/Platform/Microsoft.Testing.Extensions.Telemetry/AppInsightsProvider.cs(87,9): error CS0103: (NETCORE_ENGINEERING_TELEMETRY=Build) The name '_task' does not exist in the current context

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 14, 2026

@copilot address review comments and fix the following issues:

@azure-pipelines
...

Addressed in a450882.

  • Fixed AppInsightsProvider build errors by removing the invalid _task = task; assignment and qualifying RoslynDebug as Microsoft.Testing.Platform.RoslynDebug.
  • Updated AppInsightsProviderTests to use DisposeAsync() only on NETCOREAPP and Dispose() otherwise, so the test compiles for non-NETCOREAPP targets too.
  • Verified with:
    • dotnet build src/Platform/Microsoft.Testing.Extensions.Telemetry/Microsoft.Testing.Extensions.Telemetry.csproj -c Debug
    • dotnet test --project test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Microsoft.Testing.Extensions.UnitTests.csproj -c Debug --filter AppInsightsProviderTests

Copilot AI requested a review from Evangelink May 14, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants