dotMemory Unit not working - tests runs but without dotMemory support
Hi,
so today i was trying out dotMemory unit to see how it works.
However it seems like im not able to make it work.
I believe that im setting up everything correctly based on the guides....
Using dotMemory Unit Standalone Launcher - Help | dotMemory Unit (jetbrains.com)
I have the following test class which im trying to run:
Imports JetBrains.dotMemoryUnit.Kernel
Imports JetBrains.dotMemoryUnit
Namespace Model
<TestClass()>
Public Class MyObjectTests
<TestCategory("Memory")>
<TestMethod>
Public Sub TestObject_VerifyMemoryRelease()
' frame1 Is DotMemoryUnit.Support() method; frame2 Is "test" method
DotMemoryUnitController.TestStart(New StackTrace().GetFrame(2).GetMethod())
Console.WriteLine($"dotMemoryApi.IsEnabled = {dotMemoryApi.IsEnabled}.")
If Not dotMemoryApi.IsEnabled Then
Assert.Inconclusive("No DotMemory support.")
End If
'
' Arrange.
'
Dim objVariable = New object
Dim disposeActionInIsolation =
New Action(Sub()
Dim tobj = New TestObject(objVariable )
tobj.Dispose()
End Sub)
dotMemory.Check(Sub(memory)
Dim count = memory.GetObjects(Function(where) where.Type.
Is(Of TestObject).ObjectsCount
Console.WriteLine($"There are {count} TestObjectinstances loaded in memory.")
Assert.AreEqual(1, count)
End Sub)
'
' Act.
'
disposeActionInIsolation()
'
' Assert.
'
GC.Collect()
Console.WriteLine($"GC.Collect.")
dotMemory.Check(Sub(memory)
Dim count = memory.GetObjects(Function(where) where.Type.
Is(Of TestObject).ObjectsCount
Console.WriteLine($"There are {count} TestObjectinstances loaded in memory.")
Assert.AreEqual(0, count)
End Sub)
DotMemoryUnitController.TestEnd()
End Sub
<TestCategory("Memory")>
<TestMethod>
Public Sub dotMemoryApiCheck()
' frame1 Is DotMemoryUnit.Support() method; frame2 Is "test" method
DotMemoryUnitController.TestStart(New StackTrace().GetFrame(2).GetMethod())
Console.WriteLine($"dotMemoryApi.IsEnabled = {dotMemoryApi.IsEnabled}.")
Assert.Fail()
DotMemoryUnitController.TestEnd()
End Sub
<TestCategory("Memory")>
<TestMethod>
Public Sub TestObject_VerifyMemoryRelease_IntentionalFail()
' frame1 Is DotMemoryUnit.Support() method; frame2 Is "test" method
DotMemoryUnitController.TestStart(New StackTrace().GetFrame(2).GetMethod())
Console.WriteLine($"dotMemoryApi.IsEnabled = {dotMemoryApi.IsEnabled}.")
If Not dotMemoryApi.IsEnabled Then
Assert.Inconclusive("No DotMemory support.")
End If
'
' Arrange.
'
Assert.IsTrue(False)
Dim objVariable = New object
Dim tobj = New TestObject(objVariable )
'
' Act.
'
tobj.Dispose()
'
' Assert.
'
GC.Collect()
dotMemory.Check(Sub(memory)
Assert.AreEqual(0,
memory.GetObjects(Function(where) where.Type.
Is(Of TestObject).ObjectsCount)
End Sub)
DotMemoryUnitController.TestEnd()
End Sub
End Class
End Namespace
Then i compile my test and try to run it with dotmemoryunit.exe :
./dotMemoryUnit.exe "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe" --propagate-exit-code --log-level=INFO -- C:\mypath\test.UnitTests\bin\Debug\test.UnitTests.dll
and then the tests seems to run but with the inconclusive status since dotMemoryApi.IsEnabled = False
This is the output:
Starting test discovery, please wait...
A total of 1 test files matched the specified pattern.
Skipped TestObject_VerifyMemoryRelease [344 ms]
Data collector 'UnitTestIsolation' message: No Microsoft .NET applications were launched during the test run. Launch applications from Explorer or a new command prompt after a test case has started. If no .NET application is being tested, disable the Test Impact and IntelliTrace diagnostic data adapters..
Data collector 'UnitTestIsolation' message: No Microsoft .NET applications were launched during the test run. Launch applications from Explorer or a new command prompt after a test case has started. If no .NET application is being tested, disable the Test Impact and IntelliTrace diagnostic data adapters..
Data collector 'UnitTestIsolation' message: No Microsoft .NET applications were launched during the test run. Launch applications from Explorer or a new command prompt after a test case has started. If no .NET application is being tested, disable the Test Impact and IntelliTrace diagnostic data adapters..
Failed dotMemoryApiCheck [4 ms]
Error Message:
Assert.Fail failed.
Stack Trace:
at dotMemoryApiCheck() in ... :line 1003
Standard Output Messages:
dotMemoryApi.IsEnabled = False.
Skipped TestObject_VerifyMemoryRelease_IntentionalFail [4 ms]
Test Run Failed.
Total tests: 3
Failed: 1
Skipped: 2
Total time: 4.8015 Seconds
Not sure if those warning are what cause the framework to not work...
As you can see i also tried to add the recommended "unsupported unit test framework" code in order to use vstest.console.exe which is what is now being used to run MS unit tests (mstest being depricated) (still wondering why is vstest.console.exe not supported?).
what am i doing wrong?
Thank you for the help.
Please sign in to leave a comment.
Hello,
MSTest is included to the Supported Unit Testing Frameworks (jetbrains.com) list. You don't need to call DotMemoryUnitController.TestStart and DotMemoryUnitController.TestEnd manually from your test, because it'll be done automatically for MSTest. When you call DotMemoryUnitController.TestStart(New StackTrace().GetFrame(2).GetMethod()) in your tests, the dotMemory Unit will try to start profiling when it's already running, and you'll get the following warning:
This error occurs because DotMemoryUnitController.TestStart is actually called twice.
Here is the full output of your test suite:
If I remove all DotMemoryUnitController calls, the tests run correctly:
I can't reproduce your case when dotMemoryApi.IsEnabled returns false when running from the command line via dotMemoryUnit.exe vstest.console.exe. My project references .net 5.0 and dotMemory Unit 3.2 and the edited code is here:
Hi,
thank you for your reply.
First of I just want to make sure that im using vstest.console.exe and not mstest. Maybe you guys see it as the same thing but just wanted to make sure it is ok since it is not explicitly listed in the supported framework.
However, I want to add that initially I didn't had the TestStart or TestEnd statement. I added those because the tests were not working and have the same outcome no matter if i remove them.
I suspect I have something that is not setup properly but unsure what as I believe I follow the manuals correctly.
One thing though, i am using .NET Framework 4.8 which is not listed in the supported framework. Could that be the issue? If so why wouldn't it be supported since all other framework except this one is listed?
Here is the outcome i get when running the tests without the TestStart and TestEnd
As you can see my tests seems to be skipped since they fail at the following statement:
If Not dotMemoryApi.IsEnabled Then
Assert.Inconclusive("No DotMemory support.")
End If
and as you can see the dotMemoryApiCheck test will return that the dotMemoryApi.IsEnabled is actually returning false.
Thank you for your help.
One other thing, when i run my tests using vstest.console.exe directly inside powershell, all works well. My other test pass correctly.
However when run them using dotMemoryUnit.exe all my test which are using shims/fakes seems to be getting errors regarding the profiler which cannot be found. I guess this is quite important for DotMemory to behave properly:
Failed MyTestWithShim [3 ms]
Error Message:
Test method MyTestWithShimTest threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to get profiler module handle 'C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\CommonExtensions\Microsoft\IntelliTrace\Microsoft.IntelliTrace.Profiler.dll'. The specified module could not be found ---> System.ComponentModel.Win32Exception: The specified module could not be found
Stack Trace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.LibraryMethods.GetModuleHandle(String fileName)
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String profilerPath)
--- End of inner exception stack trace ---
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.LoadProfilerModule(String profilerPath)
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.Initialize()
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
Hi,
We cannot reproduce your case where dotMemoryApi.IsEnabled returns False in .NET Framework 4.8 project. Could you please upload your project sample and specify again how you run tests under the dotMemory Unit?
Could you please also upload a project sample for this case?
You can upload files to our ftp.
Hi again Anna,
i managed to find the cause of the issue. It seems like DotMemoryUnit.exe was not running fine when running inside my Dowload folder which i have extracted the files.
Once i changed the path to use DotMemoryUnit.exe found in the nuget install folder everything stated to work properly.
Thank you for the time you took investigating this issue.
Thank you for information.
Do not hesitate to contact us if you have any other questions.
actually i may have another issue which you might help with.
Now that I've managed to make my test run correctly locally, I tried to make them run part of my CI azure devops pipeline.
So I created a task which basically run a Powershell script which will automatically find the latest dotMemoryUnit.exe found in the nuget package location.
However when i run my test im getting the following same errors as initially on my machine:
I tried running them manually with an elevated powershell and still the same error.
I also tried putting the dotMemoryUnit files in my agent working folder with the same issue.
So knowing that my previous issue, i was wondering if there was some kind of requirements on where to install dotMemoryUnit files and what is required to it to run successfully?
Also i was wondering if there is a better way to call dotMemoryunit in a azure devops pipeline other then going through a powershell or other script?
Thanks
Quick update. I found the cause of the issue and it seem to be a problem with DotMemoryUnit when fake dlls are present in the folder.
I will build an example project to show the issue.
I reply with it later today
Hello Eric,
Thank you for the update. Project sample will really help us to investigate it. You can upload any files to our ftp.
Hi Anna,
sure i have completed the sample project replicating the issue. The file was uploaded to your FTP with ID=2023_02_23_BbJL8z9oLrQ6jRQtCyoYqV
I have included a readme to explain the bug. Basically as soon as there is 1 *.fakes.dll present in the folder where the test is run, dotmemoryunit.exe seems to fail to load its profiling session.
here is the content of my readme.txt to explain the issue further more:
I forgot to mention, that this problem also occurs even if the test dll was produced without any fakes defined in it.
Just to conclude, I have modified my CI Script to delete any files that ends with *.Fakes.dll
Now azure task runs correctly and my memory tests are run as expected.
Also, im posting my powershell script task that i used for my CI task in case this might help anyone else in the community:
Hello Eric,
Thank you for the information. We've created corresponding ticket in our tracker: https://youtrack.jetbrains.com/issue/DMU-281/dotMemory-Unit-not-working-with-the-Fakes
Please use the workaround (delete any files that ends with *.Fakes.dll) to run dotMemory Unit tests.