Can CUs with Event Subscribers on high traffic tables causing high memory usage at NST?

KURN

Hi guys,

as in the topic asked: Is it possible that one CU with many Event Subscribers on high traffic tables (like Sales Line, Item Ledger Entry) causing a high memory usage on NST?

We noticed that we have one little process which do the following: On every change on the table (insert/modify), write a little log to another customer table, if a condition match. This event is placed on Sales Line, Item Ledger Entry, Capacity Ledger Entry, Transfer Line and a few more.

Pseudo Code:

Subscriber CU (e. a. 50000):

[EventSubscriber] T32_OnAfterModify(VAR Rec : Record "Item Ledger Entry";VAR xRec : Record "Item Ledger Entry";RunTrigger : Boolean)
LoggingCU.log('blabla');

[EventSubscriber] T37_OnAfterDelete(VAR Rec : Record "Sales Line";RunTrigger : Boolean)
LoggingCU.log('blabla');

[EventSubscriber] T37_OnAfterModify(VAR Rec : Record "Sales Line";VAR xRec : Record "Sales Line";RunTrigger : Boolean)
LoggingCU.log('blabla');

... and so on (at all around 10 subscribers).


When we check our NST (we have 16 at all with load balancing) - we see that the memory usage is slowly growing after a time from 1 GB up to 8 GB each NST.

IF this CU is causing this issue... would it help to set it on SingleInstance? This would mean if the CU is fired it ll keep on the same instance and do not load a new instance to the memory on every call.... right?

Its NAV 2016 on SQL 2014. Hardware for all components is very powerful (maybe here or there a bad setting, but we still work on it).

This is just one theory... in same time we are also checking other possibilities in our whole architecture.

Would be happy about any opinion or experience

Best regards
Kurn

*edited.. holy moly.. NST .. not NAS

SilverX

Well, that is not that easy to answer. I remember we had an issue with NAV 2015 memory leaks caused by Report Viewer and Shared variable usage inside RDLC. This was a bit mitigated with NAV 2017 I think, which has started to execute Report Viewer in a different AppDomain.

But I have no idea if this was ever backported, I assume not. If you want to put your focus on Reports, test it with an isolated instance on a test database, automatically running the most frequented reports multiple hundred times. If you see increasing memory (after all application objects have been loaded), this is most likely the mentioned issue. We were able to get every NST to around 9-10 GB of memory cunsumption, until we saw a significant performance drop.

The only tool we used was Task Manager that time, but there are many better out there. For general memory/leak analysis, I find this is a good start: Find, fix, and avoid memory leaks

SilverX

I would check the SI theory first. Should be easy and not breaking anything (if only subscribers in the CU) .

It could be RDLC reports heavily leaking memory. (this is theory 2).

SilverX

Regarding RDLC memory leaks: https://cloudblogs.microsoft.com/dynamics365/no-audience/2016/05/20/rdlc-report-and-performance-in-microsoft-dynamics-nav-2015-and-2016/

KURN

Thanks for your reply and your ideas.

We changed the affected CU to single instance and observed the behavior of the NST memory for the last two weeks. Without restarting the NSTs we noticed that the memory again is growing up to 12 GB each. This looks just not healthy in my opinion, even if we have a heavily modified system.

So with keeping an eye on heavy lifting reports... any idea how to identify the objects/processes which fill the NST memory? Is there a tool or way to see which process takes how much memory? Maybe also if this process is keeping the memory up instead of releasing it again?

Thanks in advance

Best regards
Kurn

SilverX

Well, that is not that easy to answer. I remember we had an issue with NAV 2015 memory leaks caused by Report Viewer and Shared variable usage inside RDLC. This was a bit mitigated with NAV 2017 I think, which has started to execute Report Viewer in a different AppDomain.

But I have no idea if this was ever backported, I assume not. If you want to put your focus on Reports, test it with an isolated instance on a test database, automatically running the most frequented reports multiple hundred times. If you see increasing memory (after all application objects have been loaded), this is most likely the mentioned issue. We were able to get every NST to around 9-10 GB of memory cunsumption, until we saw a significant performance drop.

The only tool we used was Task Manager that time, but there are many better out there. For general memory/leak analysis, I find this is a good start: Find, fix, and avoid memory leaks

KURN

Thanks for your reply! that looks very intersting and i will give it a try.
And yes - the first approach would be to focus on reports.