Respect working directory of 'dotnet test' when determining results directory#7265
Conversation
4a74ebf to
5fd14ec
Compare
nohwnd
left a comment
There was a problem hiding this comment.
None of the tests are testing with the new variable, they just provide empty environment.
|
@copilot resolve the merge conflicts in this pull request |
…otnettest-cwd # Conflicts: # src/Platform/Microsoft.Testing.Platform/Configurations/ConfigurationManager.cs # src/Platform/Microsoft.Testing.Platform/Services/ExecutableInfo.cs Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
…environment in tests Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Merge conflicts resolved. Two files had conflicts:
Fixed in commit
|
There was a problem hiding this comment.
Pull request overview
Updates Microsoft.Testing.Platform configuration/logging to base the default TestResults directory on the dotnet test working directory (via a new environment variable), rather than only the test host’s current working directory.
Changes:
- Add support for
DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORYwhen computing the default results directory (and diagnostic log default directory). - Thread
IEnvironmentthroughConfigurationManager→AggregatedConfigurationto read the new environment variable. - Update and extend unit tests to cover the new results-directory base behavior.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/MSTest.Engine.UnitTests/Adapter_ExecuteRequestAsyncTests.cs | Updates AggregatedConfiguration construction to include IEnvironment. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/TestApplicationBuilderTests.cs | Updates AggregatedConfiguration construction to include IEnvironment. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/Configuration/ConfigurationManagerTests.cs | Updates ConfigurationManager construction to include IEnvironment. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/Configuration/AggregatedConfigurationTests.cs | Adds coverage for DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY affecting default TestResults base. |
| src/Platform/Microsoft.Testing.Platform/TestHostControllers/TestHostControllersManager.cs | Removes now-unused configuration-derived working directory retrieval. |
| src/Platform/Microsoft.Testing.Platform/Services/CurrentTestApplicationModuleInfo.cs | Uses explicit new ExecutableInfo(...) construction. |
| src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.cs | Passes IEnvironment into the updated ConfigurationManager and removes unused working-directory code. |
| src/Platform/Microsoft.Testing.Platform/Helpers/EnvironmentVariableConstants.cs | Introduces DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY constant. |
| src/Platform/Microsoft.Testing.Platform/Configurations/ConfigurationManager.cs | Adds IEnvironment dependency and forwards it to AggregatedConfiguration. |
| src/Platform/Microsoft.Testing.Platform/Configurations/AggregatedConfiguration.cs | Uses the new env var as the base for default results directory calculation. |
| src/Platform/Microsoft.Testing.Platform/Builder/TestApplication.cs | Uses the new env var as the base for the default diagnostic log directory. |
Copilot's findings
- Files reviewed: 11/11 changed files
- Comments generated: 2
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Test Expert Reviewer 🧪
Date: 2026-05-11
Repository: microsoft/testfx
Key Findings
The PR introduces good coverage for the new DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY env-var feature. The two new tests (IndexerTest_DotnetCliTestCommandWorkingDirectorySet_UsedAsResultDirectoryBase and CheckTestResultsDirectoryOverrideAndCreateItAsync_DotnetCliTestCommandWorkingDirectorySet_UsedAsResultDirectoryBase) are well-structured and verify the core happy path.
However, three issues are worth addressing:
-
[Coverage — important] No test verifies that
--results-directorytakes precedence overDOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORYwhen both are present. The production code has an explicit comment explaining this priority is critical for the Retry extension, but there's no regression guard. -
[Structure — minor] The async env-var test sets up
mockTestApplicationModuleInfostubs that are never called (dead setups), because the env-var path short-circuits before the module-info members are needed. -
[Structure — pre-existing, minor] Two existing tests (
_GetDirectoryFromCommandLineProviderand_GetDirectoryFromStore) are byte-for-byte identical — one of them appears to be a mislabeled duplicate or an incomplete test.
Recommendations
- Add a test that sets both
DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORYand--results-directoryand asserts the CLI option wins. - Remove the dead
mockTestApplicationModuleInfosetups fromCheckTestResultsDirectoryOverrideAndCreateItAsync_DotnetCliTestCommandWorkingDirectorySet_UsedAsResultDirectoryBase. - Investigate (and fix) the two identical
CheckTestResultsDirectoryOverrideAndCreateItAsync_ResultsDirectoryIsNull_Get*tests.
Generated by Test Expert Reviewer
🧪 Test quality reviewed by Test Expert Reviewer 🧪
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: pr-expert-reviewer
Date: 2026-05-11
Repository: microsoft/testfx
Key Findings
[Correctness – Medium] Asymmetry between PlatformResultDirectory and PlatformCurrentWorkingDirectory
GetResultsDirectoryCore now uses DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY as the base for the result folder, but GetCurrentWorkingDirectoryCore does not. After this change, in any dotnet test scenario where no --results-directory is given:
config[PlatformResultDirectory]→{dotnet_test_cwd}/TestResultsconfig[PlatformCurrentWorkingDirectory]→{app_binary_dir}(or config-provider value)
Extensions that derive paths relative to PlatformCurrentWorkingDirectory (coverage files, attachments, TRX custom locations) will land in a different directory tree than the main results. This breaks the implicit contract that results files live under a common root. See inline comment for the precise location.
Positive Observations
- Correct priority ordering is maintained:
--results-directoryCLI arg still wins over the new env var, which wins over configuration providers, which wins over the default. No regression in the priority chain. - Diagnostic logger path receives the same fix via
CreateFileLoggerIfDiagnosticIsEnabled, so diagnostic logs go alongside results even before configuration is built — this is the right level to apply it given the init order. - Dead code removal in
TestHostBuilder.csandTestHostControllersManager.cs(ApplicationStateGuard.Ensure(currentWorkingDirectory is not null)) is safe:GetCurrentWorkingDirectoryCore()always returns non-null due to its fallback chain, so these assertions could never fail. Removing them is correct. - Test coverage for the new env-var behavior is present in
AggregatedConfigurationTests— both the indexer path andCheckTestResultsDirectoryOverrideAndCreateItAsyncare exercised. nameofpattern for the new constant is correct and self-validating.
Recommendations
- Decide whether
PlatformCurrentWorkingDirectoryshould also shift toDOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORYwhen the env var is set, or document the intentional asymmetry with a code comment so extension authors are not surprised. - (Minor/optional) Consider adding a
Path.IsPathRootedguard when reading the env var — ifdotnet testever sends an empty or relative value,Path.Combine("", "TestResults")silently produces a relative path resolved from the process CWD, which is harder to diagnose than an early validation failure.
Generated by pr-expert-reviewer 🧠
🧠 Reviewed by Expert Code Reviewer 🧠
|
@copilot address review comments |
…kingDirectory and improve tests Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Addressed in commit Production code (
Tests (
|
…pace Treat empty/whitespace values of DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY as not set (normalize to null) so Path.Combine does not produce relative paths. Applied in GetResultsDirectoryCore, GetCurrentWorkingDirectoryCore, and CreateFileLoggerIfDiagnosticIsEnabled.
… GetResultsDirectoryCore PlatformCurrentWorkingDirectory (via GetCurrentWorkingDirectoryCore) already incorporates the env var, so the separate check in GetResultsDirectoryCore was redundant and could cause the default results directory to ignore an explicitly-set working directory or a provider-supplied one.
The env var leaks from the parent dotnet test process into child test host processes, causing DiagnosticTests to resolve the TestResults directory relative to the CI working directory instead of the test application directory.
|
@copilot resolve the merge conflicts in this pull request |
…otnettest-cwd Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Merge conflicts resolved. The only conflict was in |
|
@copilot address review comments |
…CLI_TEST_COMMAND_WORKING_DIRECTORY Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Addressed in the latest commit. Two changes:
|
Depends on dotnet/sdk#52570
DOTNET_CLI_TEST_COMMAND_WORKING_DIRECTORY(set bydotnet test) is now used as the base directory for bothPlatformResultDirectory(defaulting to{cwd}/TestResults) andPlatformCurrentWorkingDirectory/PlatformTestHostWorkingDirectory, preserving the invariant that all output paths share a common root when running underdotnet test. The--results-directoryCLI argument continues to take precedence over the environment variable.