dotCover and xUnit not gathering coverage statistics in some environments
We're using dotCover 2.7.2 and xUnit.net 1.9.2.
On my machine (Windows 7) and a co-worker's machine (Windows 8) we can run dotCover from the command line against one of our unit test assemblies that uses xUnit.net and get correct coverage reporting.
On our build machine (Windows Server 2008 R2 Standard) when we run the same thing the only code coverage that dotCover reports is the unit test assembly itself.
We're running xUnit.net using the MSBuild task. Here's the relevant pieces from the .msbuild file.
<UsingTask TaskName="Xunit.Runner.MSBuild.xunit" AssemblyFile="$(PackagesDir)xunit.$(XunitVersion)\lib\net20\xunit.runner.msbuild.dll" />
<Target Name="XunitTests">
<xunit Assembly="$(TrunkDir)src.UnitTests\Ipseity.Server.Events.UnitTests\bin\Debug\Ipseity.Server.Events.UnitTests.dll" />
</Target>
We're running dotCover from the command line using the following command (same command in each environment).
>"c:\Program Files (x86)\JetBrains\dotCover\v2.7\Bin\dotCover.exe" analyse /TargetExecutable="c:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe" /TargetArguments="/t:XunitTests ipseity.msbuild" /TargetWorkingDir=c:\tfs\SI\ipseity\trunk\build /Output=c:\temp\coverage.xml
On the Windows 7 and Windows 8 machines the coverage.xml file contains the following.
<Root CoveredStatements="1977" TotalStatements="7867" CoveragePercent="25" ReportType="Xml" DotCoverVersion="2.7.2.176">
<Assembly Name="Ipseity.Server.Common" CoveredStatements="4" TotalStatements="339" CoveragePercent="1">
...
</Assembly>
<Assembly Name="Ipseity.Server.Events" CoveredStatements="691" TotalStatements="798" CoveragePercent="87">
...
</Assembly>
<Assembly Name="Ipseity.Server.Events.UnitTests" CoveredStatements="1240" TotalStatements="1251" CoveragePercent="99">
...
</Assembly>
<Assembly Name="ipseity.Server.MessageProcessing" CoveredStatements="42" TotalStatements="5479" CoveragePercent="1">
...
</Assembly>
</Root>
However on the build server (Windows Server 2008 R2 Standard) we only get the unit test assembly showing up in the coverage report.
<Root CoveredStatements="1033" TotalStatements="1039" CoveragePercent="99" ReportType="Xml" DotCoverVersion="2.7.2.176">
<Assembly Name="Ipseity.Server.Events.UnitTests" CoveredStatements="1033" TotalStatements="1039" CoveragePercent="99">
...
</Assembly>
</Root>
At this point we're baffled as to why we get different results on the build server than on our dev boxes so any suggestions as to what else to look for would be appreciated.
Please sign in to leave a comment.
Hello Craig,
Sorry for the delay in replying.
Please do the following on your build server:
· clear %TEMP% folder
· run coverage analysis with these two additional command line parameters: /KeepIntermediateData /LogFile
· send us zipped content of %TEMP% along with new snapshots
Kind regards,
Maria
I just wanted to follow up and ensure you received the information I sent via email and if you had had any response yet?
Hello Craig,
Could you please tell me what e-mail address you sent this information to?
Kind regards,
Maria
It was "Maria Serebryansky <devnet@jetbrains.com>". Basically I just replied to the email notice that I got. I just noticed there is an option to attach files to the post so I've attached the same archive that I emailed.
Attachment(s):
Archive.zip
Hi,
It's been a couple of weeks since I uploaded the zip file, I was wondering if there was any news on this issue. If there is additional information you require please let me know what and how to get the info and I'm happy to provide it.
Hello Craig,
I apologize for the delay in replying. We've studied the logs. According to the information there, coverage analysis runs correctly and gathered data is complete. We need some additional information. Can we schedule a remote debugging session on your machine?
Kind regards,
Maria
Let me check with my management to see if a remote debugging session is possible. Just so I have complete information, would you need to install any additional software on the machine? I'm assuming you would need interactive (i.e. logon) access to the machine?
Our developer will connect to your machine via TeamViewer. For communication we normally use Skype. So you need the following tools to be installed:
That will be enough for now.
Kind regards,
Maria
Sorry for the delay in getting back to you, I've been trying to track down the right people to get permission to do this.
I have downloaded the TeamViewer Quick Support module and tested that someone outside our network can access the machine remotely.
Skype will be a bit of a problem, we're not allowed to install external IM software on our computers. Can we use regular phone for communication? I have a phone at my desk with a direct dial number. I can also use the web-based gmail IM if that would work.
I am free most of the day Wednesday and Friday this week. Next week Monday, Tuesday, Thursday and Friday are all pretty open.
I'm in Portland, Oregon and am available from 7:30am until 5:30pm PST most days.
Hello Craig,
I've sent you an e-mail.
Kind regards,
Maria
The issue appeared to be related to shadow copying.
xUnit MSBuild task performs shadow copy by default.
PDBs are not copied during the process. Whithout PDBs dotCover can't collect complete statistics for the assemblies.
Changing task call to the following one helps:
<xunit Assembly="..." ShadowCopy="False" />
That fixed the problem. We are now getting code coverage statistics in our TeamCity build. Thank you very much.
I'm pretty sure this will fix the same problem for me, except you don't say where in hundreds of project, code and config files this line belongs.
Can you give me a clue?
You just need to make sure that xunit runner doesn't use shadow copying.
In the example above xunit tests were configured and executed via msbuild task from xunit.runner.msbuild.dll