Incomplete coverage report file-set when running DotCover from the command-line

Hello David,

Sorry for the late reply. Could you please use the 'cover' command in order to get a binary snapshot and upload it to our ftp server: ftp://ftp.intellij.net/.uploads/? This would really help us investigate the problem.  

 

Kind regards,
Maria

I have uploaded (I think!) the result from the coverage command as requested.
Please let me know if you have received the file.
Arieso.Ericsson.Tests.dll.Coverage.dat.zip

 I've received the file. Thank you. I'll let you know as soon as we have  more information on this issue.

Kind regards,
Maria

Is there any news on this issue yet?  It has been two weeks now since I submitted the requested analysis file.

Hello David,

Thank you for your patience. We've studied the snapshot and analyzed the interaction between dotCover and Sonar. As a result, we are planning to communicate with Sonar team and suggest using xml format of reports. Regarding your issue, we would like to offer you a new build of dotCover with extended logging; this build will be ready in a few hours.



Kind regards,

Maria

Hello David,

Please install and try this new build of dotCover: http://download.jetbrains.com/dotcover/dotCoverSetup.2.7.2.149.msi

If you experience the same problem, then please do the following:

  · enable logging by starting Visual Studio with the following command line arguments: /dotCover.LogLevel VERBOSE /dotCover.LogFile (log files will be saved in %Temp%\JetLogs folder with auto-generated names)

  · send us a new snapshot along with log files

  · specify the name of classes for which sources are missing



Kind regards,

Maria

Hi Maria,

Thanks for the reply.
I have downloaded the version of DotCover you suggested and re-run my tests.  As far as I can see the problem still remains.
I have uploaded a zip file (DotCoverProblem.zip) to your FTP server containing the following:

• The HTML coverage report files produced by an analyse command
• The log files produced by DotCover

Please see the ReadMe.txt file in the root of the archive.

Please acknowledge receipt of the archive file.

Regards
David Razzetti

Hi David,
We've received the archive, thank you.
However to investigate the issue in addition to log files we need a coverage snapshot (as requested in the previous message).
Please also note that snapshot and logs should be taken during the same run.

Regards,
Ruslan

Hi Ruslan,

I don't understand.  I included the HTML coverage snapshot produced by executing an Analyse command in the CoverageReport folder within the zip archive.  Is this not what you need? If you need something else, please clarify.

Regards
David Razzetti

Coverage snapshot is a binary file, the main storage of dotCover data.
It is possible to generate different reports from the given snapshot.
Html report (which you've attached) is just a user-friendly data representation, but you can't do anything with it except of viewing.
Opening in VisualStudio, merging, highlighting, navigation etc. is only available for snapshots.

Please see the following topic in help:
http://www.jetbrains.com/dotcover/webhelp/What_is_Coverage_Snapshot.html

It's possible to save coverage snapshot from VS using corresponding item in the coverage results toolbar.
If you use console runner, see "cover" command:
http://www.jetbrains.com/dotcover/webhelp/dotCover__Console_Runner_Commands.html#cover

Hi Ruslan,
I've just uploaded file DotCoverProblem2.zip.
Hopefully this contains what you require.
Let me know if it doesn't.

Regards
David Razzetti

Hello,

Is there any progress on this issue?
Do you at least know what causes it yet?

Regards
David Razzetti

Hello David,
I'm sorry for the delayed response.
According to the details you've provided, it looks like your solution has external binary references to assemblies built on another machine.
Html report consists of two parts:
- coverage statistics for all assemblies loaded during coverage analysis (tree on the left side)
- optional highlighting info for the available source files (document on the right side)
Source file is considered available if it is present on the local machine or is available on the source server (according to info from the PDBs)

The issue is that Sonar relies on parsing of this optional highlighting details (intended for presentation purposes only) to import coverage statistics.
This approach is not correct. There is a set of xml-based reports which better suit this purpose.
Generally speaking, the issue you're experiencing is a result of incorrect data import, which should be resolved on Sonar side.

However, if you'd like to get a workaround, please ensure that
- all assemblies (especially the ones you'd like to get coverage results for) are built on the machine where coverage analysis takes place
- source files are not removed right after the build

If you'd like to get some codebase-specific details related to the snapshot you've sent, please let us know your email.

Regards,
Ruslan

Hi Ruslan,
Thank you for your reply.

I don't really follow your argument as to why I have detailed coverage for some classes and not for others.
I was looking in detail at the coverage for a single quite large assembly where the assembly was built on the local PC, so an associated PDB file was available for it.
Some classes contained within the assembly had complete coverage, others contained within the same assembly did not.  Certainly the assembly references other third-party assemblies for which no PDB information is available.

Similarly I don't see how DotCover within Visual Studio 2013 manages to produce a complete coverage report for the assembly in question, but the command-line tool does not.

Please send your detailed analysis of the code-base to:
david dot razzetti at jdsu dot com
or
david dot razzetti563 at btinternet dot com

Regards
David Razzetti

Hi David,
You should get coverage statistics for all classes.
Source code highlighting (which takes place in html report) will be available only if source file is available.

If you will compare html reports generated from console and from VS, they should be the same.
As well as if you will generate coverage snapshot in console, open it in VS and compare to the snapshot generated in VS, they will also be the same.

I've sent you a list of folder names where source files are missing by e-mail.

Regards,
Ruslan

Hi Ruslan,

As far as I see I have issues similar to David's with dotCover report + SonarQube. At the current moment I was managed to get HTML report with dotCover (running with command line).
All the time I used it for a single assembly in order to avoid problems with merging reports and my report's structure looks similar to example provided on github (https://github.com/SonarSource/sonar-examples/blob/master/projects/languages/csharp/dotCover.html). If I open this report in any browser it provides me details about coverage (tree view). After clicking on namespaces, class name etc. (on treeview) I have a 'No source available.' message on the right panel. BUT if I expand the tree and click on METHOD name - I have source code with highlighting on the right panel. Which means that it's a valid report as I see.
Next, I'm trying to pass this report to SonarQube with:
sonar.cs.dotcover.reportsPaths="***path***\dotCover.html"
but still don't have coverage result on SonarQube (open related project -> Time Machine link -> Coverage.
I have attached screenshot about an empty coverage widget I'm talking about.

So, my question is: how to make dotCover report and load it into SonarQube to get results in 'Coverage' widget?

P.S. I met this issue on some other forums but it was not resolved at the current moment. Maybe I missed something.

Hi Ruslan,

The behaviour that I see is not consistent with your statements.  In an attempt to verify your statements, I have reviewed my operations.  Here is what I have done and what I have observed.

For the purposes of this exercise, I am interested in a single class.  It is called GpehNormaliser and it lives in an assembly named Arieso.Ericsson.Umts.dll.  The source file for this class (and all of the other classes that reside directly within the assembly) are located on my local PC.  This assembly is built on my local PC using Visual Studio 2013.  I have verified that the project file specifies a build with full debug information.  The PDB file for the assembly is written to the output folder containing the assembly during the build.  So on my local PC I have the C# source file for the class, the assembly containing the class and the assembly's PDB file.

I have run DotCover from the command line using the 'cover' command.  I used the NUnit console runner as the target executable and instructed NUnit to run some unit tests (contained in a separate test assembly) that exercise the code within the Arieso.Ericsson.Umts.dll assembly, including the GpehNormaliser class.

I then ran DotCover from the command line using the 'report' command to produce an HTML coverage report.  When I open the HTML file within a browser and navigate to the class of interest I can see that the class has 53% coverage.  When I attempt to drill-down and view the coverage for a particular method with source highlighting, I get the "No source available" message.  According to your previous reply, I have met all of the criteria required in order to have such highlighting for this class.

Here are some more observations based on a more detailed inspection of the HTML coverage report:
The LHS of the report provides a list of assemblies, ordered by the degree of coverage of the classes within.  Starting from the top and working down, I have source highlighting in the assemblies that I would expect to have it in (ones where the source files, assembly and PDB are all present on the local PC) and no highlighting in other assemblies that were built elsewhere.  The presence of source highlighting where expected stops abruptly broadly two thirds of the way down the ordered list.  Above a certain point there is the expected source highlighting, below the point there is none.

The first assembly with no source highlighting is one that should have it; the source, assembly and PDB are all local and the assembly was built locally at the same time that the other assemblies were built.  The assembly in question is large, containing over 400 classes.

This behaviour suggests to me a problem with the command-line tool-set.  Where do we go from here?

Regards
David Razzetti

Hi.
Source files are associated only with the leaf nodes of the coverage tree in html report.
It is expected that source files will not be shown for root, solution folder, assembly and namespace nodes.
As it was previously said html report is not intended to be used for data import.
We are going to solve this issue with SonarQube team.
The results will be published in this thread.

Regards,
Ruslan

Hi David.
Can we schedule a remote debug session on your machine?

Regards,
Ruslan

I have replied directly via email.

David,
Thank you for the assistance.
I can confirm that there was also an issue on our side.
https://youtrack.jetbrains.com/issue/DCVR-7128
Fix will be included into one of the next updates.