What is retaining ".NET, total" memory

Hello Mark,

.NET used is the amount of memory in the managed heap used by the app. This is the only part of memory .NET allows you to work with. .NET used = Gen0(used) + Gen1(used) + Gen2(used) + LOH(used)
All .NET used objects are presented in the snapshot.

.NET total is a sum of heaps size. It's the amount of memory where the objects are placed. This region can be fragmented and can contain free parts of memory. dotMemory snapshot contains .NET total value which is usually more than .NET used value.

As we can see on your screenshots, some objects were released after GC but .NET total memory didn't change. Probably heap contains free memory parts after GC, you can check it on "Inspections" view (click on snapshot name to open): https://www.jetbrains.com/help/dotmemory/Heap_Fragmentation.html?Wave=182

You can find Heap fragmentation diagram at the bottom of view, please check "free" memory values to verify this supposition.

One more possible case is a finalization queue. You can find objects that were queued for finalization on "Finalizable objects" inspection view which is also available from "Inspections" view. But this case is unlikely, since these objects must be finalized after several GCs. Free memory regions is more possible cause in your case.

Hello Anna,

Thanks for the Comment. However I still cannot find any cause, something in my code that I could change or re-arrange. Depending on the size of the database of signals that I load, the program's overall memory requirements are increasing by hundreds of megabytes! This never decreases, and I cannot figure this out. Such a large leak should be EASY to find.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

"As we can see on your screenshots, some objects were released after GC but .NET total memory didn't change. Probably heap contains free memory parts after GC, you can check it on "Inspections" view (click on snapshot name to open): https://www.jetbrains.com/help/dotmemory/Heap_Fragmentation.html?Wave=182

You can find Heap fragmentation diagram at the bottom of view, please check "free" memory values to verify this supposition."

Here is a screenshot of Snapshot 4, "root" (where the "objects" and ".NET, used" memory correctly decreases but ".NET, total" does not decrease):

How does that help me further?
What do you mean with please check "free" memory values to verify this supposition?? Where do you suggest I look? There is nothing to click on in the "Heap Fragmentation" sub-view.

"One more possible case is a finalization queue. You can find objects that were queued for finalization on "Finalizable objects" inspection view which is also available from "Inspections" view. But this case is unlikely, since these objects must be finalized after several GCs. Free memory regions is more possible cause in your case."

Not sure what to check here.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

In this screenshot, again from Snapshot 4 "root", I show the "Finalizable objects":

Objects of class AttributeRow1" and "AttributeViewModel2New + RowAtt" are exactly the ones that are instantiated, and then cleared, when I switch from Snapshot 3 to 4. 

What is this telling me? I'm clearing and NULLing all of these objects, directly in code. I have no real external resources to the .NET program (just CLI / C++ which apparently is managed). If I click on "AttributeViewModel2New + RowAtt", I see this:

And under that:

i.e. no help at all.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Here is another observation: The "Unmanaged Memory" Display in the main view. Here are 4 small screenshots from the main profiling page, corresponding to the "start", "load", "signal", & "root" Snapshots:

What is this telling me? I've not found a good description for what you refer to as "Unmanaged Memory" and what it entails.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Here is a better question: do you do live debugging sessions, where we use a tool like TeamViewer or Skype to share my screen, and we debug together? Probably 1/2 hour is all we would need with an expert like yourself. That would be great and probably much more effective. Could that work?

If you do not do that, do you know anyone who would??

Thanks,
Mark

Mark,

Here is the answers on your questions.

> Here is a screenshot of Snapshot 4, "root" (where the "objects" and ".NET, used" memory correctly decreases but ".NET, total" does not decrease):

Snapshot 4 contains a lot of free memory in the heap:
200,1 KB (LOH) + 73,70MB (Gen2) + 3,44MB (Gen1) = 77,34 MB free memory

When the objects are released by the garbage collector the memory goes into the "free" heap space. .NET keeps these memory regions to make future allocation of objects and increase performance of allocations in future.

"Free" heap space should be released back to the OS if physical memory is low. This is how .NET framework works and we can't affect this behavior. Thus, it's not a memory leak.

> There is nothing to click on in the "Heap Fragmentation" sub-view.

You don't need to open detailed generations info to get information about "free" memory. This memory doesn't contain any objects, so information from "Inspections" view is enough. Оtherwise, use "Generations" view to get information about how objects are distributed among generations.

> In this screenshot, again from Snapshot 4 "root", I show the "Finalizable objects"

It looks like a real performance issue in your application. You have a lot of objects that are queued for finalization. Finalizable objects are the objects that use the Finalize() method to release unmanaged resources. The problem of using this pattern is, first, that the lifetime of finalizable objects is extended by at least one more GC cycle and, second, that the finalization thread (that executes the Finalize() method) is run unpredictably. This may cause problems in case you want to reclaim the released resources as quickly as possible and may lead to sudden performance drops.
We suggest to support Dispose Pattern for all objects with finalizers: https://docs.microsoft.com/en-us/dotnet/standard/design-guidelines/dispose-pattern

Snapshot doesn't contain stack trace for finalizable objects (screenshot at the moment when you click on "AttributeViewModel2New + RowAtt") because collecting allocations data was disabled. It always disabled if you attach profiler to already running process. Оtherwise, you should enable flag "Start collecting allocation data immediately" in profiler options when you configure your profiling session or click "Collect Allocations" button to enable data collecting during profiling session.

> I'm clearing and NULLing all of these objects, directly in code

It's a bad coding practice. Managed objects must be released by GC when they are no longer needed. NULLing can hide a real issues. Please try to use Dispose Pattern as mentioned above.

> I've not found a good description for what you refer to as "Unmanaged Memory" and what it entails.

Unmanaged memory is the memory allocated outside of the managed heap and not managed by Garbage Collector. Generally, this is the memory required by .NET CLR, dynamic libraries, graphics buffer, and so on. This part of memory cannot be analyzed in the profiler.