dotCover performance (6 times slower that normally)

Hi.
I'm expiriencing very low performance of dotCover. For a unit test (running by mstest) I have 0:27 min with no dotCover used and 3:12 min with dotCover.
When I run the test with MSTest code coverage I get 0:35.
Sometimes difference is even more. I'm also using coverage when running TeamCity build. There I get even more worse result: 1:07 min vs 9:49 min (with dC). Just imagine the impact of using dotCover for ~4000 tests! Terrible.

Is it expected performance loss for dotCover using?

7 comments

Hello Shike,
It is expected that running tests with enabled coverage takes longer than simple run.
This is because of the code instrumentaion which makes coverage analysis possible.
Regarding the comparison with MSTest code coverage:
- please note that by default dotCover collects coverage info for all assemblies, while MSTest only for the fixed set of assemblies defined in "testrunconfig".
You can reduce scope of coverage analysis with coverage filters in dotCover as described here:
http://blogs.jetbrains.com/dotnet/2010/07/filtering-with-dotcover
- besides total statistics dotCover also collects per-test info
(You can see which code is covered by the each particular test and for every piece of code you can get list of tests which cover it)
http://blogs.jetbrains.com/dotnet/2010/07/show-covering-test-with-dotcover/
It also takes some time.

dotCover is regularly tested on the large solutions (with more than 30K tests)
Overall impact for them results in the increase of the execution time in about 2-3 times.
We continuously work on the performance improvements aiming to make coverage analysis as close to the non-instrumented execution in terms of consumed time as possible.

0

Thanks for the answer, Ruslan.
Just for your information. Running a TeamCity-build with dotCover (with enabled filter) takes 4 hours, but running it with NCover takes 2.5 h. NCover has default settings without any assembly filters.
Unfortunately it's far beyond 2-3 minutes.
"Show covering tests" feature is owesome indeed but overall performance isn't.
I suspect I'm experiencing some perf loss you aren't aware of.
It may help: I have many tests with inheritence where there's a base class with [TestClass] attribute and several inherited classes:
[TestClass]
public class MyComponentTests
{
    [TestMethod]
    public void DoingSomeThingShouldThrow() {}
}
[TestClass]
public class MyComonentViaWcfTests
{
}
here I have two test methods actually.

Thanks.

0

Hello Shike,
Thank you for the feedback.
Some comments:
- regarding filters I meant not NCover but MSTest coverage
- execution difference with normal run is not 2-3 minutes, but 2-3 times.
(it is always a subject for continuous improvements)
We'll provide additional checks for your usecase as well.

0

Ok, sorry for my misunderstaing about 2-3 times.
TeamCity doesn't support MSTest's CC so I have to use NCover for comparing with dotCover.
I wish you to find areas for improvements of this very nice tool.

0

Version 2018.1.1 still 6 times slower. 6h vs 1h for 1880 NUnit integration tests, understandable when instrumenting, but is it really not possible to improve performance of how the code is instrumented from command line? (I use /Scope=)

0

Please sign in to leave a comment.