Nonpaged pool memory leak mfeavfk.sys on multiple windows server 2012 R2

[bodysoda]

We have ENS 10.6.1.1068 installed across 600 windows servers. We found 25% of servers Non-paged pool memory leaks poolmon tag with MFe0 which points to mfeavfk.sys consuming high memory. Gradually server hungs due runs out of memory and reboot is carried with no other options. 

Task manager shows: 

When you start pool mon with /p switch for non paged and the sort by bytes (b)  .. you can see the usage quite clearly the MFeS drivers taking the memory

Rammap shows 

As per the known issues KB82450 please note I dont have any debug logging turned. 

After uninstall the ENS from the problematic server and reboot the server's memory seems to be stable. However we can't run production system with AV. Anyone with ENS 10.6.1 expereience this issue?

[akatt]

@bodysoda 

10.6.1.1068 is the original release of the Common Platform. 

Reference:
https://kc.mcafee.com/corporate/index?page=content&id=KB82761

We are aware of memory leaks reported with this version (and as it pertains to the Threat Prevention module, Exploit Prevention, and Adaptive Threat Protection), some of which were addressed within the December release.

Reference:  https://kc.mcafee.com/resources/sites/MCAFEE/content/live/PRODUCT_DOCUMENTATION/27000/PD27825/en_US/...

Problem is that we pulled the update package for December, but kept the full installer available.  We are also aware that not all reported memory leaks are fully resolved with the December update.  In order to install all available fixes from the full installer package for December, when already using 10.6.1, it will most likely require a removal, reboot, and then clean install using that latest full installer package.

What might  prove more beneficial, is to wait for the February update, and then simply apply that update (it will come with all previous fixes as well).

We would also be very insterested in receiving a data set showing the memory leak, when using the latest available code, so opening a support service request would be ideal.

Was my reply helpful?

If this information was helpful in any way, or answered your question, will you please select "Accept as Solution" in my reply, or give kudos as appropriate, so together we can help other members?

[bodysoda]

@akatt, Thanks for the update and acknowledgment of memory leak issues. I've logged support ticket and will provide reference number in private chat.

[bodysoda]

@akatt, is there any way to restart the McAfee Service Controller so that memory is relased back to OS? Lot of servers in my environment is hung/unresponsive as they sitting 100% memory in couple of days. It Any workaround to address the memory issue untill feb patch release?

[akatt]

@bodysoda 

The product won't allow us to manually restart the service, even if we disable the common platform self-protection, and also Threat Prevention's Access Protection.

Disabling Access Protection, however, will allow us to "end task" on the McAfee processes using Task Manager.  For example, we can end task on McShield, and McShield will then start again on its own.  Provided this is the process showing the unexpectedly high memory consumption, it should help alleviate the symptoms for a bit.  Could also do the same with mfetp.exe, or mfeatp.exe.  

Since we are aware of some memory leaks being manifested by the ATP module, we could consider uninstalling ATP for the time being as well, but it might not be necessary since February's update should be here today.

Was my reply helpful?

If this information was helpful in any way, or answered your question, will you please select "Accept as Solution" in my reply, or give kudos as appropriate, so together we can help other members?

[jess_arman · Accepted solution]

@bodysoda ENS 10.6.1 has had multiple monthly updates (Nov, Dec, Feb) that have addressed causes for memory leaks. I would advise that you upgrade one system to ENS 10.6.1 February update to confirm if your particular leak resolves. 

In the event it does not resolve for some reason, or alters in behavior, then we would ask that you please follow the data collection instructions found in KB86691 for investigation of a memory leak, and then open a case with Support in order to work with them to determine the cause or potentially file a defect.

Was my reply helpful?

If this information was helpful in any way, or answered your question, will you please select "Accept as Solution" in my reply, or give kudos as appropriate, so together we can help other members?

[bodysoda]

@jessica1608, @akatt , so I applied the Feb patch goes in and stops NPP memory leak , but NPP isn’t cleared till the server restarts. 

[chealey · Accepted solution]

I am glad to hear that the Feb Update has fixed the issue! The fact you need to reboot is expected behaviour. The memory can't be cleared until a reboot occurs so if you are seeing a memory leak issue, a reboot is required.

[User16096767]

Thank you for the detailed description of the issue. We just started seeing the same issue in many of our  servers.

We had to upgraded to ENS 10.6.1 November release to address Memory leask from 10.2.1. It looks like we are back it again.

[bodysoda]

Glad you found the post helpful. 

Don't be shy to give kudos on the post you found helpful