Skip to content

Apply structured assertion messages (RFC 012) to AreEqual / AreNotEqual #8231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Evangelink wants to merge 19 commits into
base: main
Choose a base branch
from

Merge origin/main and resolve conflicts

1950cf2
Select commit
Loading
Failed to load commit list.
Open

Apply structured assertion messages (RFC 012) to AreEqual / AreNotEqual #8231

Merge origin/main and resolve conflicts
1950cf2
Select commit
Loading
Failed to load commit list.
Azure Pipelines / microsoft.testfx failed May 16, 2026 in 40m 4s

Build #20260516.50 had test failures

11 errors / 7 warnings

Details

Tests

  • Failed: 8 (0.04%)
  • Passed: 20,615 (99.04%)
  • Other: 192 (0.92%)
  • Total: 20,815

Annotations

Check failure on line 175 in src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs#L175 

src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs(175,1): error : [ScopeWithSingleFailure_TestFails] [net11.0] Assertion failed. Expected condition to be true.
Expression 'AssertOutputMatchesRegex' failed for member 'ScopeWithSingleFailure_TestFails' at line 30 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs'. Output of the test host is:
Command: /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests --filter ScopeWithSingleFailure
====================
ExitCode: 2
====================
StandardOutput:
MSTest v4.3.0-ci (UTC 5/16/2026) [osx-x64 - .NET 10.0.8]
failed ScopeWithSingleFailure (765ms)
  Assertion failed. Expected values to be equal.
  expected: 1
  actual:   2
  Assert.AreEqual(1, 2)
    at Microsoft.VisualStudio.TestTools.UnitTesting.AssertScope.Dispose() in /_/src/TestFramework/TestFramework/Assertions/AssertScope.cs:72
    at UnitTest1.ScopeWithSingleFailure() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/UnitTest1.cs:23
    at UnitTest1.ScopeWithSingleFailure() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/UnitTest1.cs:20
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Test run summary: Failed! - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests.dll (net10.0|x64)
  total: 1
  failed: 1
  succeeded: 0
  skipped: 0
  duration: 2s 306ms
====================
StandardError:


actual: false

Assert.IsTrue(Regex.IsMatch(testHostResult.StandardOutput, pattern))

Check failure on line 454 in src/TestFramework/TestFramework/Assertions/Assert.Contains.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

src/TestFramework/TestFramework/Assertions/Assert.Contains.cs#L454 

src/TestFramework/TestFramework/Assertions/Assert.Contains.cs(454,1): error : [TrxReport_WhenTestFails_ContainsExceptionInfoInOutput ("net10.0")] [net11.0] Assert.Contains failed. String '<?xml version="1.0" encoding="utf-8"?>
<TestRun id="5e85878b-8c14-4d66-979b-beaf878a762f" name="@iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941 2026-05-16 15:07:34.2949970" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2026-05-16T15:07:29.993541Z" queuing="2026-05-16T15:07:29.993541Z" start="2026-05-16T15:07:29.993541Z" finish="2026-05-16T15:07:34.296197Z" />
  <TestSettings name="default" id="a77e95e2-520d-4edd-a08b-ac730f558f00">
    <Deployment runDeploymentRoot="_iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941_2026-05-16_15_07_34.2949970" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testId="18f2dd82-fede-8c47-821a-9500badc6846" testName="FailingTest" computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" duration="00:00:00.0825942" startTime="2026-05-16T15:07:30.4131450+00:00" endTime="2026-05-16T15:07:30.5700520+00:00" testType="13CDC9D9-DDB5-4fa4-A97D-D965CCFC6D4B" outcome="Failed" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" relativeResultsDirectory="d5d6e0dd-dbc9-461b-873c-a9639d831be7">
      <Output>
        <ErrorInfo>
          <Message>Assertion failed. Expected values to be equal.

expected: 1
actual:   2

Assert.AreEqual(1, 2)</Message>
          <StackTrace>   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 298
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 142
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 48
   at MSTestTrxReport.UnitTest1.FailingTest() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/UnitTest1.cs:line 11
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)</StackTrace>
        </ErrorInfo>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="FailingTest" storage="/users/runner/work/1/s/artifacts/tmp/debug/testsuite/azx7c/mstesttrxreport/bin/release/net10.0/mstesttrxreport.dll" id="18f2dd82-fede-8c47-821a-9500badc6846">
      <Execution id="d5d6e0dd-dbc9-461b-873c-a9639d831be7" />
      <TestMethod codeBase="/Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/bin/Release/net10.0/MSTestTrxReport.dll" adapterTypeName="executor://MSTestExtension/4.3.0-ci" className="MSTestTrxReport.UnitTest1" name="FailingTest" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="18f2dd82-fede-8c47-821a-9500badc6846" executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Failed">
    <Counters total="1" executed="1" passed="0" failed="1" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <RunInfos>
      <RunInfo computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" outcome="Error" timestamp="2026-05-16T15:07:34.321181">
        <Text>Exit code indicates failure: '2'. Please refer to https://aka.ms/testingplatform/exitcodes for more information.</Text>
      </RunInfo>
    </RunInfos>
  </ResultSummary>
</TestRun>' does not contain string 'Assert.AreEqual failed. Expected:&lt;1&gt;. Actual:&lt;2&gt;.'. 'substring' expression: '"Assert.AreEqual failed. Expected:&lt;1&gt;. Actual:&lt;2&gt;."', 'value' expression: 'trxContent'. <?xml version="1.0" encoding="utf-8"?>
<TestRun id="5e85878b-8c14-4d66-979b-beaf878a762f" name="@iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941 2026-05-16 15:07:34.2949970" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2026-05-16T15:07:29.993541Z" queuing="2026-05-16T15:07:29.993541Z" start="2026-05-16T15:07:29.993541Z" finish="2026-05-16T15:07:34.296197Z" />
  <TestSettings name="default" id="a77e95e2-520d-4edd-a08b-ac730f558f00">
    <Deployment runDeploymentRoot="_iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941_2026-05-16_15_07_34.2949970" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testId="18f2dd82-fede-8c47-821a-9500badc6846" testName="FailingTest" computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" duration="00:00:00.0825942" startTime="2026-05-16T15:07:30.4131450+00:00" endTime="2026-05-16T15:07:30.5700520+00:00" testType="13CDC9D9-DDB5-4fa4-A97D-D965CCFC6D4B" outcome="Failed" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" relativeResultsDirectory="d5d6e0dd-dbc9-461b-873c-a9639d831be7">
      <Output>
        <ErrorInfo>
          <Message>Assertion failed. Expected values to be equal.

expected: 1
actual:   2

Assert.AreEqual(1, 2)</Message>
          <StackTrace>   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 298
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 142
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 48
   at MSTestTrxReport.UnitTest1.FailingTest() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/UnitTest1.cs:line 11
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)</StackTrace>
        </ErrorInfo>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="FailingTest" storage="/users/runner/work/1/s/artifacts/tmp/debug/testsuite/azx7c/mstesttrxreport/bin/release/net10.0/mstesttrxreport.dll" id="18f2dd82-fede-8c47-821a-9500badc6846">
      <Execution id="d5d6e0dd-dbc9-461b-873c-a9639d831be7" />
      <TestMethod codeBase="/Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/bin/Release/net10.0/MSTestTrxReport.dll" adapterTypeName="executor://MSTestExtension/4.3.0-ci" className="MSTestTrxReport.UnitTest1" name="FailingTest" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="18f2dd82-fede-8c47-821a-9500badc6846" executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Failed">
    <Counters total="1" executed="1" passed="0" failed="1" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <RunInfos>
      <RunInfo computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" outcome="Error" timestamp="2026-05-16T15:07:34.321181">
        <Text>Exit code indicates failure: '2'. Please refer to https://aka.ms/testingplatform/exitcodes for more information.</Text>
      </RunInfo>
    </RunInfos>
  </ResultSummary>
</TestRun>.

Check failure on line 259 in src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs#L259 

src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs(259,1): error : [HangDump_TemplateFileName_CreateDump ("net10.0")] [net11.0] Assert.AreEqual failed. Expected:<7>. Actual:<0>. 'expected' expression: 'exitCode', 'actual' expression: 'testHostResult.ExitCode'. Expression 'AssertExitCodeIs' failed for member 'HangDump_TemplateFileName_CreateDump' at line 121 of file '/_/test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HangDumpTests.cs'. Output of the test host is:
Command: /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/bin/Release/net10.0/HangDump --hangdump --hangdump-timeout 8s --hangdump-filename {pname}_{pid}_{tfm}_{time}_hang.dmp --results-directory /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/07438228cdd44e8f9471c72fcfb10b55/net10.0
====================
ExitCode: 0
====================
StandardOutput:
Microsoft.Testing.Platform v2.3.0-ci (UTC 05/16/2026) [osx-x64 - net10.0]
Hang dump timeout of '00:00:08' expired
Dumping process 80670 - HangDump
Creating dump file '/Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/07438228cdd44e8f9471c72fcfb10b55/net10.0/HangDump_80670_net10.0_2026-05-16_14-58-40.9076110_hang.dmp'
[createdump] Gathering state for process 80670 
[createdump] Writing full dump to file /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/07438228cdd44e8f9471c72fcfb10b55/net10.0/HangDump_80670_net10.0_2026-05-16_14-58-40.9076110_hang.dmp
[createdump] Written 5644349736 bytes (1378015 pages) to core file
[createdump] Target process is alive
Test run summary: Passed! - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/bin/Release/net10.0/HangDump.dll (net10.0|x64)
  total: 2
  failed: 0
  succeeded: 2
  skipped: 0
  duration: 2m 53s 011ms
[createdump] Dump successfully written in 159323ms
  Out of process file artifacts produced:
    - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/1WqnK/AssetFixture/07438228cdd44e8f9471c72fcfb10b55/net10.0/HangDump_80670_net10.0_2026-05-16_14-58-40.9076110_hang.dmp
====================
StandardError:

Check failure on line 7274 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

Build log #L7274 

Bash exited with code '2'.

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails 

Assertion failed. Expected condition to be true.
Expression 'AssertOutputMatchesRegex' failed for member 'ScopeWithSingleFailure_TestFails' at line 30 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs'. Output of the test host is:
Command: /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/WmODQ/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests --filter ScopeWithSingleFailure
====================
ExitCode: 2
====================
StandardOutput:
MSTest v4.3.0-ci (UTC 05/16/2026) [linux-x64 - .NET 10.0.8]
failed ScopeWithSingleFailure (92ms)
  Assertion failed. Expected values to be equal.
  expected: 1
  actual:   2
  Assert.AreEqual(1, 2)
    at Microsoft.VisualStudio.TestTools.UnitTesting.AssertScope.Dispose() in /_/src/TestFramework/TestFramework/Assertions/AssertScope.cs:72
    at UnitTest1.ScopeWithSingleFailure() in /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/WmODQ/SoftAssertionTests/UnitTest1.cs:23
    at UnitTest1.ScopeWithSingleFailure() in /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/WmODQ/SoftAssertionTests/UnitTest1.cs:20
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Test run summary: Failed! - /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/WmODQ/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests.dll (net10.0|x64)
  total: 1
  failed: 1
  succeeded: 0
  skipped: 0
  duration: 678ms
====================
StandardError:


actual: false

Assert.IsTrue(Regex.IsMatch(testHostResult.StandardOutput, pattern))
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertIsTrueFailed(Nullable`1 condition, String message, String conditionExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs:line 175
   at Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers.AcceptanceAssert.AssertOutputMatchesRegex(TestHostResult testHostResult, String pattern, String callerMemberName, String callerFilePath, Int32 callerLineNumber) in /_/test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/Helpers/AcceptanceAssert.cs:line 87
   at MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails() in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs:line 30
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput ("net10.0")

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput ("net10.0") 

Assert.Contains failed. String '<?xml version="1.0" encoding="utf-8"?>
<TestRun id="fbcf16f0-b94c-456d-8c7a-784ad6a739ce" name="@b96b181dc000000 2026-05-16 14:51:28.3108826" ns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2026-05-16T14:51:27.9667132Z" queuing="2026-05-16T14:51:27.9667132Z" start="2026-05-16T14:51:27.9667132Z" finish="2026-05-16T14:51:28.3117671Z" />
  <TestSettings name="default" id="098aeb3d-18dd-4b7e-bebd-3d6d6cc019de">
    <Deployment runDeploymentRoot="_b96b181dc000000_2026-05-16_14_51_28.3108826" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="192c918b-a046-4527-a919-ecfb7a7b6d1b" testId="18f2dd82-fede-8c47-821a-9500badc6846" testName="FailingTest" computerName="b96b181dc000000" duration="00:00:00.0457987" startTime="2026-05-16T14:51:28.1256581+00:00" endTime="2026-05-16T14:51:28.2268957+00:00" testType="13CDC9D9-DDB5-4fa4-A97D-D965CCFC6D4B" outcome="Failed" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" relativeResultsDirectory="192c918b-a046-4527-a919-ecfb7a7b6d1b">
      <Output>
        <ErrorInfo>
          <Message>Assertion failed. Expected values to be equal.

expected: 1
actual:   2

Assert.AreEqual(1, 2)</Message>
          <StackTrace>   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 298
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 142
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 48
   at MSTestTrxReport.UnitTest1.FailingTest() in /mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/C9q2X/MSTestTrxReport/UnitTest1.cs:line 11
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)</StackTrace>
        </ErrorInfo>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="FailingTest" storage="/mnt/vss/_work/1/s/artifacts/tmp/debug/testsuite/c9q2x/mstesttrxreport/bin/release/net10.0/mstesttrxreport.dll" id="18f2dd82-fede-8c47-821a-9500badc6846">
      <Execution id="192c918b-a046-4527-a919-ecfb7a7b6d1b" />
      <TestMethod codeBase="/mnt/vss/_work/1/s/artifacts/tmp/Debug/testsuite/C9q2X/MSTestTrxReport/bin/Release/net10.0/MSTestTrxReport.dll" adapterTypeName="executor://MSTestExtension/4.3.0-ci" className="MSTestTrxReport.UnitTest1" name="FailingTest" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="18f2dd82-fede-8c47-821a-9500badc6846" executionId="192c918b-a046-4527-a919-ecfb7a7b6d1b" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Failed">
    <Counters total="1" executed="1" passed="0" failed="1" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <RunInfos>
      <RunInfo computerName="b96b181dc000000" outcome="Error" timestamp="2026-05-16T14:51:28.3298436">
        <Text>Exit code indicates failure: '2'. Please refer to https://aka.ms/testingplatform/exitcodes for more information.</Text>
      </RunInfo>
    </RunInfos>
  </ResultSummary>
</TestRun>' does not contain str
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Contains(String substring, String value, String message, String substringExpression, String valueExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs:line 454
   at MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput(String tfm) in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/TrxReportTests.cs:line 38
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput ("net10.0")

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput ("net10.0") 

Assert.Contains failed. String '<?xml version="1.0" encoding="utf-8"?>
<TestRun id="5e85878b-8c14-4d66-979b-beaf878a762f" name="@iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941 2026-05-16 15:07:34.2949970" ns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2026-05-16T15:07:29.993541Z" queuing="2026-05-16T15:07:29.993541Z" start="2026-05-16T15:07:29.993541Z" finish="2026-05-16T15:07:34.296197Z" />
  <TestSettings name="default" id="a77e95e2-520d-4edd-a08b-ac730f558f00">
    <Deployment runDeploymentRoot="_iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941_2026-05-16_15_07_34.2949970" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testId="18f2dd82-fede-8c47-821a-9500badc6846" testName="FailingTest" computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" duration="00:00:00.0825942" startTime="2026-05-16T15:07:30.4131450+00:00" endTime="2026-05-16T15:07:30.5700520+00:00" testType="13CDC9D9-DDB5-4fa4-A97D-D965CCFC6D4B" outcome="Failed" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" relativeResultsDirectory="d5d6e0dd-dbc9-461b-873c-a9639d831be7">
      <Output>
        <ErrorInfo>
          <Message>Assertion failed. Expected values to be equal.

expected: 1
actual:   2

Assert.AreEqual(1, 2)</Message>
          <StackTrace>   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 298
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 142
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 48
   at MSTestTrxReport.UnitTest1.FailingTest() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/UnitTest1.cs:line 11
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)</StackTrace>
        </ErrorInfo>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="FailingTest" storage="/users/runner/work/1/s/artifacts/tmp/debug/testsuite/azx7c/mstesttrxreport/bin/release/net10.0/mstesttrxreport.dll" id="18f2dd82-fede-8c47-821a-9500badc6846">
      <Execution id="d5d6e0dd-dbc9-461b-873c-a9639d831be7" />
      <TestMethod codeBase="/Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/AZX7c/MSTestTrxReport/bin/Release/net10.0/MSTestTrxReport.dll" adapterTypeName="executor://MSTestExtension/4.3.0-ci" className="MSTestTrxReport.UnitTest1" name="FailingTest" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="18f2dd82-fede-8c47-821a-9500badc6846" executionId="d5d6e0dd-dbc9-461b-873c-a9639d831be7" testListId="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8C84FA94-04C1-424b-9868-57A2D4851A1D" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Failed">
    <Counters total="1" executed="1" passed="0" failed="1" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <RunInfos>
      <RunInfo computerName="iad01-dz256-17465d49-9651-46a3-a964-2618e8fcbdc5-8AD4D9D08941" outcome="Error" timestamp="2026-05-16T15:07:34.321181">
        <Text>Exit code ind
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Contains(String substring, String value, String message, String substringExpression, String valueExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.Contains.cs:line 454
   at MSTest.Acceptance.IntegrationTests.TrxReportTests.TrxReport_WhenTestFails_ContainsExceptionInfoInOutput(String tfm) in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/TrxReportTests.cs:line 38
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails 

Assertion failed. Expected condition to be true.
Expression 'AssertOutputMatchesRegex' failed for member 'ScopeWithSingleFailure_TestFails' at line 30 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs'. Output of the test host is:
Command: /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests --filter ScopeWithSingleFailure
====================
ExitCode: 2
====================
StandardOutput:
MSTest v4.3.0-ci (UTC 5/16/2026) [osx-x64 - .NET 10.0.8]
failed ScopeWithSingleFailure (765ms)
  Assertion failed. Expected values to be equal.
  expected: 1
  actual:   2
  Assert.AreEqual(1, 2)
    at Microsoft.VisualStudio.TestTools.UnitTesting.AssertScope.Dispose() in /_/src/TestFramework/TestFramework/Assertions/AssertScope.cs:72
    at UnitTest1.ScopeWithSingleFailure() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/UnitTest1.cs:23
    at UnitTest1.ScopeWithSingleFailure() in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/UnitTest1.cs:20
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Test run summary: Failed! - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/ChotE/SoftAssertionTests/bin/Release/net10.0/SoftAssertionTests.dll (net10.0|x64)
  total: 1
  failed: 1
  succeeded: 0
  skipped: 0
  duration: 2s 306ms
====================
StandardError:


actual: false

Assert.IsTrue(Regex.IsMatch(testHostResult.StandardOutput, pattern))
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertIsTrueFailed(Nullable`1 condition, String message, String conditionExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs:line 175
   at Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers.AcceptanceAssert.AssertOutputMatchesRegex(TestHostResult testHostResult, String pattern, String callerMemberName, String callerFilePath, Int32 callerLineNumber) in /_/test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/Helpers/AcceptanceAssert.cs:line 87
   at MSTest.Acceptance.IntegrationTests.SoftAssertionTests.ScopeWithSingleFailure_TestFails() in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/SoftAssertionTests.cs:line 30
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129
View more details on Azure Pipelines