Inconsistent coverage results using dotCover console runner with .NET Core unit tests or vstest.console.exe.

Affected versions: 2018.1.1 - 2019.2.2

Symptoms

The coverage results are always 0% or change from run to run (even if there were no changes in the analyzed assemblies).

 

Possible cause

  1. dotCover 2019.2: When covering "dotnet test" on a solution level, dotCover analyzes only the first .csproj with unit tests (other projects are not analyzed).
  2. dotCover 2019.1 and earlier: When covering "dotnet test", "dotnet vstest" or vstest.console.exe (version 2017 and later), dotCover might get 0% coverage because the process with tests is killed by test platform before dotCover has a chance to save a snapshot.

 

Workaround (only for dotCover 2019.1 and earlier)

dotCover console runner:

  • You have a Windows machine with installed dotCover 2018.1.1 or later.
  • You have a .NET Core unit test project or use vstest.console.exe to run tests.
  • You want to perform coverage analysis of unit tests via the command line.

 

To perform coverage analysis:

  1. Create (or modify an existing) a .runsettings file for the project, e.g. mySettings.runsettings.
  2. Modify the file based on the example below

    <?xml version="1.0" encoding="utf-8"?>  
    <RunSettings>
     <RunConfiguration>
       <DisableParallelization>True</DisableParallelization>
     </RunConfiguration>
     <InProcDataCollectionRunSettings>
       <InProcDataCollectors>
         <InProcDataCollector friendlyName="JetBrainsProfiler" uri="InProcDataCollector://JetBrains/JetBrainsProfiler/1.0" assemblyQualifiedName="<AQN>" codebase="<PATH>">
           <Configuration />
         </InProcDataCollector>
       </InProcDataCollectors>
     </InProcDataCollectionRunSettings>
    </RunSettings>

    Here
    • <AQN>: an assembly qualified name that depends on a dotCover version. For dotCover 2018.1.1, it is: JetBrains.Common.UnitTesting.InProcDataCollector.GrouplessCoverageInProcDataCollector, JetBrains.Common.UnitTesting.InProcDataCollector, Version=112.0.0.0, Culture=neutral, PublicKeyToken=1010a0d8d6380325
    • <PATH>: an absolute path to the JetBrains.Common.UnitTesting.InProcDataCollector.dll library which is a part of dotCover. For example:
      C:\Users\[YourUserName]\AppData\Local\JetBrains\Installations/JetBrains/Installations/dotCover12/JetBrains.Common.UnitTesting.InProcDataCollector.dll

  3. Run coverage analysis from the command line with the additional /UseApi parameter.
    For example:

    dotCover.exe cover /TargetExecutable="C:\Program Files\dotnet\dotnet.exe" /TargetArguments="test -s C:\MyProject\mySettings.runsettings C:\MyProject\MyTests.csproj" /Output=snapshot.dcvr /UseApi


dotCover on TeamCity:

  • You have a TeamCity server with installed dotCover 2018.1.1 or later.
  • You have a .NET Core unit test project or use Visual Studio Tests build step.
  • You want to perform coverage analysis of unit tests via dotCover.


To perform coverage analysis:

  1. Create (or modify an existing) a .runsettings file for the project, e.g. mySettings.runsettings.
  2. Modify the file based on the example below

    <?xml version="1.0" encoding="utf-8"?>  
    <RunSettings>
     <RunConfiguration>
       <DisableParallelization>True</DisableParallelization>
     </RunConfiguration>
     <InProcDataCollectionRunSettings>
       <InProcDataCollectors>
         <InProcDataCollector friendlyName="JetBrainsProfiler" uri="InProcDataCollector://JetBrains/JetBrainsProfiler/1.0" assemblyQualifiedName="<AQN>" codebase="PATH_TO_InProcDataCollector">
           <Configuration />
         </InProcDataCollector>
       </InProcDataCollectors>
     </InProcDataCollectionRunSettings>
    </RunSettings>

    Here
    • <AQN>: an assembly qualified name that depends on a dotCover version. For dotCover 2018.1.1, it is: JetBrains.Common.UnitTesting.InProcDataCollector.GrouplessCoverageInProcDataCollector, JetBrains.Common.UnitTesting.InProcDataCollector, Version=112.0.0.0, Culture=neutral, PublicKeyToken=1010a0d8d6380325
  3. On TeamCity server add “File content replacer” to Build Features to change PATH_TO_InProcDataCollector to %teamcity.tool.JetBrains.dotCover.CommandLineTools.DEFAULT%/JetBrains.Common.UnitTesting.InProcDataCollector.dll in mySettings.runsettings file:

    2018-05-28_1630.png

  4. Run coverage analysis in “.NET CLI(dotnet)” build step with “test” command type or in “Visual Studio Tests” build step. Expand Advanced options, set a path to mySettings.runsettings and add /UseApi to the Additional arguments:

    2018-05-28_1844.png

    2018-05-28_1846.png


0 out of 0 found this helpful
7 comments

Is there any update on this?  The workaround is not great because it requires updating the file whenever changing versions of DotCover on TC.

1

Is any fix planned for this? In addition, it would be useful it the article listed the InProcDataCollector assembly version for different versions of dotCover.

2
Avatar
Permanently deleted user

The fix will be available in the upcoming 2019.2 release.

0

Thank you!

0

Hi, I cannot tell from the release notes if it was actually fixed:

https://confluence.jetbrains.com/display/TW/TeamCity+2019.1.2+%28build+66342%29+Release+Notes

 

Please reply with clarification.

Edited by Darren Sherwood
0
Avatar
Permanently deleted user

Darren,

The issue was fixed on the dotCover's side, not in TeamCity. dotCover 2019.2 automatically adds all required parameters to the test runner command-line when covering .NET Core or VSTest tests. It's not bundled with the current TeamCity version yet but you can install it manually.

0

Thanks!

I can confirm that it works.

0

Please sign in to leave a comment.

Have more questions?

Submit a request