Skip to content

Apply structured assertion messages (RFC 012) to Assert.Fail / Assert.Inconclusive #8296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Evangelink merged 11 commits into from
May 18, 2026

Fix failing assertion message expectations

024ea6a
Select commit
Loading
Failed to load commit list.
Merged

Apply structured assertion messages (RFC 012) to Assert.Fail / Assert.Inconclusive #8296

Fix failing assertion message expectations
024ea6a
Select commit
Loading
Failed to load commit list.
Azure Pipelines / microsoft.testfx failed May 17, 2026 in 40m 54s

Build #20260517.33 had test failures

12 errors / 8 warnings

Details

Tests

  • Failed: 10 (0.05%)
  • Passed: 21,016 (99.05%)
  • Other: 192 (0.90%)
  • Total: 21,218

Annotations

Check failure on line 27164 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

Build log #L27164 

Cmd.exe exited with code '2'.

Check failure on line 218 in test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs#L218 

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs(218,1): error : [AreAllDistinct_Generic_WithComparer_HasDuplicate_ShouldFail] [.NET Framework 4.8] Expected exception message to match the equivalent of "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)", but "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests+CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)" does not.

Check failure on line 271 in test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs#L271 

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs(271,1): error : [AreAllDistinct_NonGeneric_WithComparer_HasDuplicate_ShouldFail] [.NET Framework 4.8] Expected exception message to match the equivalent of "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   CaseInsensitiveStringComparer

Assert.AreAllDistinct(list, <comparer>)", but "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests+CaseInsensitiveStringComparer

Assert.AreAllDistinct(list, <comparer>)" does not.

Check failure on line 218 in test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs#L218 

test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs(218,1): error : [AreAllDistinct_Generic_WithComparer_HasDuplicate_ShouldFail] [net8.0] Expected exception message to match the equivalent of "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)", but "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests+CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)" does not.

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA ("net462")

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA ("net462") 

Assert.AreEqual failed. Expected:<0>. Actual:<2>. 'expected' expression: 'exitCode', 'actual' expression: 'testHostResult.ExitCode'. Expression 'AssertExitCodeIs' failed for member 'TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA' at line 27 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/STAThreadingTests.cs'. Output of the test host is:
Command: D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\bin\Release\net462\STATestThreading.exe
====================
ExitCode: 2
====================
StandardOutput:
MSTest v4.3.0-ci (UTC 5/17/2026) [x64 - .NET Framework 4.8.9325.0]
failed TestMethod1 (51ms)
  Assert.AreEqual failed. Expected:<STA>. Actual:<MTA>. 'expected' expression: 'ApartmentState.STA', 'actual' expression: 'apartmentState'.
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertFailed(String assertionName, String message)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression)
    at UnitTest1.AssertCorrectThreadApartmentState()
failed TestMethod2 (3ms)
  Assert.AreEqual failed. Expected:<STA>. Actual:<MTA>. 'expected' expression: 'ApartmentState.STA', 'actual' expression: 'apartmentState'.
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertFailed(String assertionName, String message)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression)
    at UnitTest1.AssertCorrectThreadApartmentState()
    at UnitTest1.<TestMethod2>d__1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
    at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.<ExecuteInternalAsync>d__52.MoveNext()
failed TestMethod3 (9ms)
  Assertion failed. Expected condition to be false.
  actual: true
  Assert.IsFalse(Thread.CurrentThread.IsThreadPoolThread)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertFailed(StructuredAssertionMessage structuredMessage)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertIsFalseFailed(Nullable`1 condition, String message, String conditionExpression)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsFalse(Nullable`1 condition, String message, String conditionExpression)
    at UnitTest1.TestMethod3()
Test run summary: Failed! - D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\bin\Release\net462\STATestThreading.exe (net462|x64)
  total: 3
  failed: 3
  succeeded: 0
  skipped: 0
  duration: 575ms
====================
StandardError:
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 259
   at Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers.AcceptanceAssert.AssertExitCodeIs(TestHostResult testHostResult, Int32 exitCode, String callerMemberName, String callerFilePath, Int32 callerLineNumber) in /_/test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/Helpers/AcceptanceAssert.cs:line 11
   at MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA(String tfm) in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/STAThreadingTests.cs:line 27
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129

Check failure on line 1 in MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA ("net10.0")

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA ("net10.0") 

Assert.AreEqual failed. Expected:<0>. Actual:<2>. 'expected' expression: 'exitCode', 'actual' expression: 'testHostResult.ExitCode'. Expression 'AssertExitCodeIs' failed for member 'TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA' at line 27 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/STAThreadingTests.cs'. Output of the test host is:
Command: D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\bin\Release\net10.0\STATestThreading.exe
====================
ExitCode: 2
====================
StandardOutput:
MSTest v4.3.0-ci (UTC 5/17/2026) [win-x64 - .NET 10.0.8]
failed TestMethod1 (56ms)
  Assert.AreEqual failed. Expected:<STA>. Actual:<MTA>. 'expected' expression: 'ApartmentState.STA', 'actual' expression: 'apartmentState'.
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:267
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:143
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:48
    at UnitTest1.AssertCorrectThreadApartmentState() in D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\UnitTest1.cs:59
    at UnitTest1.TestMethod1() in D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\UnitTest1.cs:12
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
failed TestMethod2 (2ms)
  Assert.AreEqual failed. Expected:<STA>. Actual:<MTA>. 'expected' expression: 'ApartmentState.STA', 'actual' expression: 'apartmentState'.
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:267
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, IEqualityComparer`1 comparer, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:143
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, String expectedExpression, String actualExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:48
    at UnitTest1.AssertCorrectThreadApartmentState() in D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\UnitTest1.cs:59
    at UnitTest1.TestMethod2() in D:\a\_work\1\s\artifacts\tmp\Debug\testsuite\gv7V8\STATestThreading\UnitTest1.cs:18
    at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:129
failed TestMethod3 (6ms)
  Assertion failed. Expected condition to be false.
  actual: true
  Assert.IsFalse(Thread.CurrentThread.IsThreadPoolThread)
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertIsFalseFailed(Nullable`1 condition, String message, String conditionExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs:236
    at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsFalse(Nullable`1 condition, String message, String conditionExpression) in /_/src/TestFramework/TestFramework/Assertions/Assert.IsTrue.cs:216
    at UnitTest1.TestMethod3() in D:\a\_work\1
Raw output 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.ReportAssertAreEqualFailed(Object expected, Object actual, String userMessage) in /_/src/TestFramework/TestFramework/Assertions/Assert.AreEqual.cs:line 259
   at Microsoft.Testing.Platform.Acceptance.IntegrationTests.Helpers.AcceptanceAssert.AssertExitCodeIs(TestHostResult testHostResult, Int32 exitCode, String callerMemberName, String callerFilePath, Int32 callerLineNumber) in /_/test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/Helpers/AcceptanceAssert.cs:line 11
   at MSTest.Acceptance.IntegrationTests.STAThreadingTests.TestMethodThreading_MainIsSTAThread_OnWindows_NoRunsettingsProvided_ThreadIsSTA(String tfm) in /_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/STAThreadingTests.cs:line 27
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.Execution.cs:line 129

Check failure on line 1 in Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_NonGeneric_WithComparer_HasDuplicate_ShouldFail

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_NonGeneric_WithComparer_HasDuplicate_ShouldFail 

Expected exception message to match the equivalent of "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   CaseInsensitiveStringComparer

Assert.AreAllDistinct(list, <comparer>)", but "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests+CaseInsensitiveStringComparer

Assert.AreAllDistinct(list, <comparer>)" does not.
Raw output 
   at AwesomeAssertions.Execution.LateBoundTestFramework.Throw(String message)
   at AwesomeAssertions.Execution.AssertionChain.FailWith(Func`1 getFailureReason)
   at AwesomeAssertions.Specialized.ExceptionAssertions`1.AssertExceptionMessage(IEnumerable`1 messages, String expectation, String because, Object[] becauseArgs)
   at AwesomeAssertions.Specialized.ExceptionAssertions`1.WithMessage(String expectedWildcardPattern, String because, Object[] becauseArgs)
   at Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_NonGeneric_WithComparer_HasDuplicate_ShouldFail() in /_/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs:line 271

Check failure on line 1 in Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_Generic_WithComparer_HasDuplicate_ShouldFail

See this annotation in the file changed.

@azure-pipelines azure-pipelines / microsoft.testfx

Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_Generic_WithComparer_HasDuplicate_ShouldFail 

Expected exception message to match the equivalent of "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)", but "Assertion failed. Expected all items in collection to be distinct.

duplicates: ["a"]
collection: ["A", "B", "a"]
comparer:   Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests+CaseInsensitiveStringComparer

Assert.AreAllDistinct(new[] { "A", "B", "a" }, <comparer>)" does not.
Raw output 
   at AwesomeAssertions.Execution.LateBoundTestFramework.Throw(String message)
   at AwesomeAssertions.Execution.AssertionChain.FailWith(Func`1 getFailureReason)
   at AwesomeAssertions.Specialized.ExceptionAssertions`1.AssertExceptionMessage(IEnumerable`1 messages, String expectation, String because, Object[] becauseArgs)
   at AwesomeAssertions.Specialized.ExceptionAssertions`1.WithMessage(String expectedWildcardPattern, String because, Object[] becauseArgs)
   at Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.AssertTests.AreAllDistinct_Generic_WithComparer_HasDuplicate_ShouldFail() in /_/test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreAll.cs:line 218
View more details on Azure Pipelines