Skip navigation
McAfee Secure sites help keep you safe from identity theft, credit card fraud, spyware, spam, viruses and online scams
6395 Views 17 Replies Latest reply: Nov 26, 2012 7:06 PM by Peacekeeper RSS 1 2 Previous Next
beheer Newcomer 44 posts since
Jul 23, 2011
Currently Being Moderated

Oct 11, 2012 8:11 AM

McAfee AntiVirus Plus 2012, Memory leak in FWPKCLNT.SYS (and terribly slow scans!)

1) Memory leak (The Nonpaged-pool fills up):

 

More than two weeks ago, after the update of all our (64-bit) Windows 7 machines, to version 11.6(SecurityCenter) and 15.6(VirusScan), excessive memory leaks occur and the scans are very slow.

After 5 to 8 days uptime (depending on the IO use), the machines crash with a BSOD(Blue Screen of Death), because the non-paged memory reaches the physical memory limit.

 

On the XP machines, 9 in total, (Version 11.0/15.0/..) these problems are not seen at all! (The *.6 seems to be a Win7 only version.)

 

The newer versions v11.6.434(SC) and v15.6.231(VS) seemed to improve the memory issue a bit, the machines build up their Nonpaged-pool slower, but did not fix the leak.

The leak occurs from the kernel-driver FWPKCLNT.SYS, in c:\Windows\Sysnative\drivers\. (On my systems it has date 2012-08-22 20:12 (UTC+1), is version: 6.1.7601.17939, and is installed on 12-September-2012, the time from the MS-Updates batch.)

This was found out using 'poolmon.exe' from the Windows Driver Kit. (The driver with Tag 'Fwpx', never Frees any! of its allocated memory and when the nonpaged-memory reaches a total of about 3GB it crashes our 4GB PCs. (Max nonpaged memory is 75% from the Physical Memory size on a Win7 64-bit machine.)

See attachement A.)

I have about 5 almost 100% identical BSODs. (All report a crash in the driver NETIO.SYS (BugCheckCode=1d, String=DRIVER_IRQL_NOT_LESS_OR_EQUAL).

See attachment B).

 

If the machines are rebooted, before the Non-paged memory reaches its border (the Nonpaged Limit), the machine does not crash.

 

Filling up almost all the Physical memory with nonpage-able pages, of cause really hurts the performance of a PC.

This is surely (one of the) reasons the next problem also occurs.

 

 

2) Slow Scanning (either Scheduled, Full or Manual):

 

From the Windows 7 PCs, all with the latest patches (so including the patches of yesterday 2012-Oct-11), 6 machines were tested.

The worst behaviour is shown on 3 (of those 6) PCs. They have a dual core processor (and only 4 GB of memory).

Also their scanning, which normally lasted maximally up to 4 hours (before this was even 2 hours!), now sometimes takes over a day or longer!

(This is even with the "Scan using minimal resources" setting turned off.)

 

The Resource Monitor (Disk-tab) shows the scan has problems with big (ASCII) files. The INBOX and Sent files from Thunderbird take a very long time to scan. During this the non-page memory size raises even more quickly.

The files are in plain ASCII text (not compressed or anything), and its size reaches up to 3.5GB for some INBOX-es. As said before, they are scanned a lot faster on the slow XP machines!

The slow scanning is occurring since it was first notices on 26-September. (We run weekly scans).

 

 

Notes/Conclusions:

 

The described problem, the memory leak, is also very extensively reported in at least 2 other threads in the forum.

It seems to be the EXACT same problem for which McAfee (until now) has not found a solution.

 

- One link which mentiones the exact same problem is the one from user Fragged, posted on 19-September:

(See: https://community.mcafee.com/thread/48661?tstart=0)

 

- It is not related only to machines with lots of IO, as the Blue Iris Camera-PCs from user Lotty.

(See: https://community.mcafee.com/thread/47463?tstart=0)

He already reported this problem as early as 7-Augustus!, yet there is no fix!

 

- It is not limited to Total Protection, as we are using VirusScan Plus only!

It seems to be in the core-software of the newer *.6 version for Windows 7 PCs.

 

- There must be a lot of unaware users from McAfee out there, who probably reboot their machines regularly, and therefore may not even notice the problem.

(Or it is simply too much hassle to write here...?)

 

 

McAfee should give this issue some very, very serious attention, because they are loosing customers this way!

 

 

 

Attachments:

 

A) Nonpaged Memory-pool usage shown by Task Manager, RamMap(sysinternals) and poolmon.

  Poolmon also shows that 0 Frees were done in the 4-days the 'Fwpx' driver build up its nonpaged pool to 2.5GB ! (NP-total=~2.9GB)

 

B) BSOD from this morning 2012-Oct-11, shown with BlueScreenView (It also shows one from 2 weeks ago!)

 

Message was edited by: beheer on 10/11/12 8:11:49 AM CDT
  • lotty Apprentice 126 posts since
    Apr 18, 2011

    Based on my direct conversations with the McAfee engineering team  …  they know they have a problem with a memory leak ... it tends to present with any significant IO event .. further the rumor is a fix has been completed but not yet released … antivirus programs .. my sense is Microsoft changed a few things in some OS updates released back in July .. McAfee did not see this coming or just did not test thoroughly enough .. McAfee released their own new build right after this July batch of Microsoft updates but failed to incorporate the changes that other antivirus vendors were able to navigate.

     

    Why hasn’t McAfee released the fix .. only they know .. but I do think they owe it to the user community to COMMUNICATE  .. .. about a month ago just prior to the release of the current builds  SC 434 / VS 231 I was told about the pending  fix .. based on my testing which I have posted in my link (see link below)  .. the current build was a  SLIGHT improvement  to the timeline of the crash event but did not change the outcome. So my conclusion was they thought they had a fix but it did not work.

     

    IF IF IF this is the case, McAfee needs to go back to the drawing board on how they test new releases. I sat  with their engineering team many times for hours while they remoted into my PCs and captured real-time logs of the traffic and crashes. McAfee documented many crash events .. throughout the process the same engineer would call me and offered insights into the current status. I have not heard from him in over a month. He did call me a couple of days before the release of the current builds to suggest I install and retest.

     

    My understanding of root cause relates to some function calls into the kernel specifically for

     

    Anyway that is my best understanding of the situation. Any yes one can only wait it out or switch. What a complete pain.

    Other links covering the same issue

     

    https://community.mcafee.com/thread/47463?tstart=0

    https://community.mcafee.com/thread/48661?tstart=0

  • Ex_Brit Volunteer Moderator 59,599 posts since
    May 6, 2004

    This is what I have received today from support and I quote:

     

    The bottom line is that this is a Microsoft problem caused by their fwpxclnt component not releasing memory consumed in Win 7, SP1 (Windows 8 doesn’t experience this problem). 

     

    This is a distilled version of our findings:

     

    ISSUE: Windows 7 SP1 runs out of memory because the nonpaged pool fills with buffers tagged Fwpx.

     

    From msdn:

    "Remarks

    The FwpsNetBufferListRemoveContext0 function asynchronously removes a tagged

    context that was previously associated by calling

    FwpsNetBufferListAssociateContext0. It is not required to use this function,

    because the tagged context will be removed automatically when the packets move

    through the stack. This function is provided so that a callout driver can stop

    processing in unusual situations."

     

    which clearly states that netio/wfp should auto-remove associated contexts.

    This is apparently not happening on win7.

     

    RESOLUTION: The network filter driver has been revised to prevent Windows 7 from allocating these buffers.

     

    (We should mention the Fwpx pool tag because it's a distinguishing feature a user might discover while troubleshooting with Poolmon or other tools.)

     

    Despite the fact this problem is *caused* by a Microsoft component, we are adjusting our software to compensate.  The change has been made and will be included in the next release of the Core components – I still haven’t heard back which specific version number that will be in, nor do I yet know when it will be released.


    https://community.mcafee.com/servlet/JiveServlet/downloadImage/102-5889-3-62127/Peter.gif
    Toronto • Canada
    Volunteer Moderator
    I can't help you privately - please post in the Forums
    Use Advanced Forum Search To Find Answers
    Beta Test McAfee Products For PC & MAC
    How To Fix File Associations in Windows
    XP & Office 2003 End-Of-Life - 08 April, 2014
    Anti-Spyware/Malware & Hijacker Tools
  • lotty Apprentice 126 posts since
    Apr 18, 2011

    FYI bobcat posting yesterday on my posting

    Got another new version tonight (11.6.435 / 15.6.231) and the problem is NOT fixed.

    postings that seem to be same issue are

    https://community.mcafee.com/thread/47463?start=90&tstart=0

    https://community.mcafee.com/thread/48661?tstart=0

    https://community.mcafee.com/thread/49170?tstart=0

  • outtasight Newcomer 6 posts since
    Nov 13, 2012

    Figures.

     

    Sometime in early October I started getting problems with a Win7 32 bit machine with 2GB RAM.  This machine is on 24x7 as a data logger and CCTV recorder.  Until then, this machine was rock solid with up-time measured in weeks (reboots only occasionally done for MS updates being applied).

     

    It started doing weird things with the machine slowing to a crawl and eventually crashing with a BSOD.  I stopped using the CCTV software for a while and it just took longer to crash (a few days).  If I caught it in the death throws, I could see the free memory stat counting down like a bomb with the disk monitor showing the system process thrashing the disk with reads and writes.  If rebooted before zero memory free, it would recover fine.

     

    I'm on the VirusScan Plus 15.6.231 and Security Center 11.6.435 with up to date MS paches.

     

    The usual pattern of: MS patches core system component... AV vendor doesn't get notified or doesn't pay attention... PC gets smegged... MS and AV vendor point fingers at each other and bury heads in sand...

     

    Definitely McAfee software needs to be able to roll back to a "Last known good" config...  Unless that version has an exploit that would allow malicious virus code to cause crashes, data loss and pain worse than the crashes, data loss and pain that the patched version of the AV product is causing me now.

     

    At least AV vendors malicious code comes with an uninstall routine

     

    Message was edited by: outtasight - got the version numbers the wrong way round on 13/11/12 01:40:44 CST
  • Peacekeeper Volunteer Moderator 21,371 posts since
    Nov 23, 2002

    Actually the fix is in 12.1 released late last week . If you uninstall mcafee and reinstall it you will get the fix. For updates it is I assume throttled and the update will take a while to come.

     

    Though 1 users says (build SC 11.6.435 / VS 15.6.231/ PFW 12.6.186) fixes it.

     

    read the last 2 pages of

    https://community.mcafee.com/message/263254#263254

     

    If you are going to uninstall do it this way

    http://service.mcafee.com/FAQDocument.aspx?lc=1033&id=TS101331

    reboot and run

    http://download.mcafee.com/products/licensed/cust_support_patches/Mcpreinstall.e xe

    reinstall from your account


    Tony
    Volunteer Moderator
    Mcafee Total Protection 7.0 beta, Windows 8 64bit
    No Unrequested PMs please
    Do you have an idea for improving McAfee products? Please share it in the new Ideas community space!  NOTE: You must register an account first.

  • outtasight Newcomer 6 posts since
    Nov 13, 2012

    Thanks.  I'll see what 12.1 brings.  I don't have time in the next couple of days to play with the machine so maybe it will get the update automatically.  Otherwise I'll try to re-install McAfee at the weekend.  I was going to dig out the last known good image backup of the machine, but if 12.1 fixes it, that will save me re-imaging the whole machine.

     

    SC 11.6.435 /  VS 15.6.231  definitely were associated with a couple of BSODs and near misses (yesterday was a near miss with the machine chugging heavily with only 4MB free RAM remaining).

     

    Selecting "update now" doesn't find anything other than todays definition update.  Another product I used had a separate "update product" option to force it to look for new software core updates as opposed to just definition updates.

  • Peacekeeper Volunteer Moderator 21,371 posts since
    Nov 23, 2002

    Oh the update is only available from the US Uk Ca and Au servers so if you are reinstalling set 1 of those as the country.


    Tony
    Volunteer Moderator
    Mcafee Total Protection 7.0 beta, Windows 8 64bit
    No Unrequested PMs please
    Do you have an idea for improving McAfee products? Please share it in the new Ideas community space!  NOTE: You must register an account first.

1 2 Previous Next

More Like This

  • Retrieving data ...

Incoming Links

Bookmarked By (0)

Legend

  • Correct Answers - 5 points
  • Helpful Answers - 3 points