Inconsistent coverage results using dotCover console runner with .NET Core unit tests or vstest.console.exe.
Affected versions: 2018.1.1 - 2019.2.2
Symptoms
The coverage results are always 0% or change from run to run (even if there were no changes in the analyzed assemblies).
Possible cause
- dotCover 2019.2: When covering "dotnet test" on a solution level, dotCover analyzes only the first .csproj with unit tests (other projects are not analyzed).
- dotCover 2019.1 and earlier: When covering "dotnet test", "dotnet vstest" or vstest.console.exe (version 2017 and later), dotCover might get 0% coverage because the process with tests is killed by test platform before dotCover has a chance to save a snapshot.
Workaround (only for dotCover 2019.1 and earlier)
dotCover console runner:
- You have a Windows machine with installed dotCover 2018.1.1 or later.
- You have a .NET Core unit test project or use vstest.console.exe to run tests.
- You want to perform coverage analysis of unit tests via the command line.
To perform coverage analysis:
- Create (or modify an existing) a .runsettings file for the project, e.g. mySettings.runsettings.
- Modify the file based on the example below
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<DisableParallelization>True</DisableParallelization>
</RunConfiguration>
<InProcDataCollectionRunSettings>
<InProcDataCollectors>
<InProcDataCollector friendlyName="JetBrainsProfiler" uri="InProcDataCollector://JetBrains/JetBrainsProfiler/1.0" assemblyQualifiedName="<AQN>" codebase="<PATH>">
<Configuration />
</InProcDataCollector>
</InProcDataCollectors>
</InProcDataCollectionRunSettings>
</RunSettings>
Here - <AQN>: an assembly qualified name that depends on a dotCover version. For dotCover 2018.1.1, it is: JetBrains.Common.UnitTesting.InProcDataCollector.GrouplessCoverageInProcDataCollector, JetBrains.Common.UnitTesting.InProcDataCollector, Version=112.0.0.0, Culture=neutral, PublicKeyToken=1010a0d8d6380325
- <PATH>: an absolute path to the JetBrains.Common.UnitTesting.InProcDataCollector.dll library which is a part of dotCover. For example:
C:\Users\[YourUserName]\AppData\Local\JetBrains\Installations/JetBrains/Installations/dotCover12/JetBrains.Common.UnitTesting.InProcDataCollector.dll - Run coverage analysis from the command line with the additional /UseApi parameter.
For example:
dotCover.exe cover /TargetExecutable="C:\Program Files\dotnet\dotnet.exe" /TargetArguments="test -s C:\MyProject\mySettings.runsettings C:\MyProject\MyTests.csproj" /Output=snapshot.dcvr /UseApi
dotCover on TeamCity:
- You have a TeamCity server with installed dotCover 2018.1.1 or later.
- You have a .NET Core unit test project or use Visual Studio Tests build step.
- You want to perform coverage analysis of unit tests via dotCover.
To perform coverage analysis:
- Create (or modify an existing) a .runsettings file for the project, e.g. mySettings.runsettings.
- Modify the file based on the example below
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<DisableParallelization>True</DisableParallelization>
</RunConfiguration>
<InProcDataCollectionRunSettings>
<InProcDataCollectors>
<InProcDataCollector friendlyName="JetBrainsProfiler" uri="InProcDataCollector://JetBrains/JetBrainsProfiler/1.0" assemblyQualifiedName="<AQN>" codebase="PATH_TO_InProcDataCollector">
<Configuration />
</InProcDataCollector>
</InProcDataCollectors>
</InProcDataCollectionRunSettings>
</RunSettings>
Here - <AQN>: an assembly qualified name that depends on a dotCover version. For dotCover 2018.1.1, it is: JetBrains.Common.UnitTesting.InProcDataCollector.GrouplessCoverageInProcDataCollector, JetBrains.Common.UnitTesting.InProcDataCollector, Version=112.0.0.0, Culture=neutral, PublicKeyToken=1010a0d8d6380325
- On TeamCity server add “File content replacer” to Build Features to change PATH_TO_InProcDataCollector to %teamcity.tool.JetBrains.dotCover.CommandLineTools.DEFAULT%/JetBrains.Common.UnitTesting.InProcDataCollector.dll in mySettings.runsettings file:
- Run coverage analysis in “.NET CLI(dotnet)” build step with “test” command type or in “Visual Studio Tests” build step. Expand Advanced options, set a path to mySettings.runsettings and add /UseApi to the Additional arguments:
Please sign in to leave a comment.
Is there any update on this? The workaround is not great because it requires updating the file whenever changing versions of DotCover on TC.
Is any fix planned for this? In addition, it would be useful it the article listed the InProcDataCollector assembly version for different versions of dotCover.
The fix will be available in the upcoming 2019.2 release.
Thank you!
Hi, I cannot tell from the release notes if it was actually fixed:
https://confluence.jetbrains.com/display/TW/TeamCity+2019.1.2+%28build+66342%29+Release+Notes
Please reply with clarification.
Darren,
The issue was fixed on the dotCover's side, not in TeamCity. dotCover 2019.2 automatically adds all required parameters to the test runner command-line when covering .NET Core or VSTest tests. It's not bundled with the current TeamCity version yet but you can install it manually.
Thanks!
I can confirm that it works.