Skip to content

Guard server telemetry against duplicate result updates per test UID #8192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Evangelink merged 4 commits into from
May 14, 2026
+52 −3
Merged

Guard server telemetry against duplicate result updates per test UID #8192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6478155
Initial plan
Copilot May 13, 2026
8f91a0d
Avoid counting duplicate server telemetry results
Copilot May 13, 2026
ebb5b5a
Clarify server telemetry duplicate-result tests
Copilot May 13, 2026
ce10af0
Rename duplicate-result telemetry tests
Copilot May 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions ...form/Microsoft.Testing.Platform/ServerMode/JsonRpc/PerRequestServerDataConsumerService.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,11 @@ private void AddOrUpdateTestNodeStateStatistics(TestNodeUid testNodeUid, bool ha
return;
}

if (existingStatistics.HasPassed == hasPassed)
Copy link
Copy Markdown
Member

@Youssef1313 Youssef1313 May 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why this is correct. Old code makes more sense to me.

If we receive two results, even if it's for the same test node, we should count it as two results.

It also doesn't make sense to gate the double counting under HasPassed == hasPassed.

If I have a folded test with two cases, imagine these two scenarios:

  • First case passed, second case failed -> we count it twice.
  • The two cases passed -> we count it once.

This doesn't make sense to me. I feel like the original issue is simply not correct and should just have been closed without making a change.

{
return;
}

if (hasPassed)
{
existingStatistics.TotalPassedRetries++;
Expand Down
50 changes: 47 additions & 3 deletions ...itTests/Microsoft.Testing.Platform.UnitTests/ServerMode/ServerDataConsumerServiceTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,12 +133,12 @@ public void PopulateTestNodeStatistics_WithDuplicatePassedEvents()
Assert.AreEqual(0, statistics.TotalDiscoveredTests);
Assert.AreEqual(1, statistics.TotalPassedTests);
Assert.AreEqual(0, statistics.TotalFailedTests);
Assert.AreEqual(1, statistics.TotalPassedRetries);
Assert.AreEqual(0, statistics.TotalPassedRetries);
Assert.AreEqual(0, statistics.TotalFailedRetries);
}

[TestMethod]
public void PopulateTestNodeStatistics_WithEventsForSameUid()
public void PopulateTestNodeStatistics_WithOutcomeChangeForSameUid_CountsAsRetry()
{
PropertyBag properties = new(
new FailedTestNodeStateProperty("failed"));
Expand All @@ -148,7 +148,7 @@ public void PopulateTestNodeStatistics_WithEventsForSameUid()
PropertyBag otherProperties = new(
PassedTestNodeStateProperty.CachedInstance);

TestNodeUpdateMessage otherTestNode = new(new SessionUid(string.Empty), new TestNode { Uid = new TestNodeUid("test()"), DisplayName = string.Empty, Properties = otherProperties });
TestNodeUpdateMessage otherTestNode = new(new SessionUid("2"), new TestNode { Uid = new TestNodeUid("test()"), DisplayName = string.Empty, Properties = otherProperties });

_service.PopulateTestNodeStatistics(testNode);
_service.PopulateTestNodeStatistics(otherTestNode);
Expand All @@ -161,6 +161,50 @@ public void PopulateTestNodeStatistics_WithEventsForSameUid()
Assert.AreEqual(0, statistics.TotalFailedRetries);
}

[TestMethod]
public void PopulateTestNodeStatistics_WithDuplicateFailedEvents_DoesNotCountAsRetry()
{
PropertyBag properties = new(
new FailedTestNodeStateProperty("failed"));

TestNodeUpdateMessage testNode = new(new SessionUid("1"), new TestNode { Uid = new TestNodeUid("test()"), DisplayName = string.Empty, Properties = properties });

_service.PopulateTestNodeStatistics(testNode);
_service.PopulateTestNodeStatistics(testNode);

TestNodeStatistics statistics = _service.GetTestNodeStatistics();
Assert.AreEqual(0, statistics.TotalDiscoveredTests);
Assert.AreEqual(0, statistics.TotalPassedTests);
Assert.AreEqual(1, statistics.TotalFailedTests);
Assert.AreEqual(0, statistics.TotalPassedRetries);
Assert.AreEqual(0, statistics.TotalFailedRetries);
}

[TestMethod]
public void PopulateTestNodeStatistics_WithDuplicateEventAfterOutcomeChange_OnlyCountsFirstRetry()
{
PropertyBag failedProperties = new(
new FailedTestNodeStateProperty("failed"));

TestNodeUpdateMessage failedTestNode = new(new SessionUid("1"), new TestNode { Uid = new TestNodeUid("test()"), DisplayName = string.Empty, Properties = failedProperties });

PropertyBag passedProperties = new(
PassedTestNodeStateProperty.CachedInstance);

TestNodeUpdateMessage passedTestNode = new(new SessionUid("1"), new TestNode { Uid = new TestNodeUid("test()"), DisplayName = string.Empty, Properties = passedProperties });

_service.PopulateTestNodeStatistics(failedTestNode);
_service.PopulateTestNodeStatistics(passedTestNode);
_service.PopulateTestNodeStatistics(passedTestNode);

TestNodeStatistics statistics = _service.GetTestNodeStatistics();
Assert.AreEqual(0, statistics.TotalDiscoveredTests);
Assert.AreEqual(1, statistics.TotalPassedTests);
Assert.AreEqual(0, statistics.TotalFailedTests);
Assert.AreEqual(1, statistics.TotalPassedRetries);
Assert.AreEqual(0, statistics.TotalFailedRetries);
}

[TestMethod]
public void PopulateTestNodeStatistics_WithEventsForDifferentUids()
{
Expand Down