Skip navigation
McAfee Secure sites help keep you safe from identity theft, credit card fraud, spyware, spam, viruses and online scams
3617 Views 10 Replies Latest reply: Mar 1, 2013 11:06 AM by jbmartin6 RSS 1 2 Previous Next
btlyric Apprentice 184 posts since
Aug 1, 2012
Currently Being Moderated

Feb 26, 2013 9:59 PM

anti-malware overloaded

We had two recent experiences where client systems received the message:

 

Anti-Malware Engine Overloaded

 

The Anti-Malware engine is currently overloaded andcontent delivery is not permitted without being checked for viruses. Please try again later.

 

One occurrence happened over the weekend. The other happened yesterday. Devices involved were not the same.

 

These occurrences seemed to be related to failed AV engine updates as mentioned in the following KB article:

 

https://kc.mcafee.com/corporate/index?page=content&id=KB76036

 

Specific errors included:

 

[Anti-Malware Engine] [ErrorFromAVEngine] Error message from engine: 'McAfee micro incremental update failed: 8'

[Anti-Malware Engine] [ErrorFromAVEngine] Error message from engine: 'McAfee Gateway Anti-Malware Engine: failed to load MFE base API in '/opt/mwg/plugin/data/antivirus/SCANM/573''

[Anti-Malware Engine] [ErrorLoadingSCANM] Failed to load and initialize 'McAfee Gateway Anti-Malware' engine from directory '/opt/mwg/plugin/data/antivirus/SCANM/573'. Error code: 'SCANMAPI_ERROR_LOADFAILED'.

[Anti-Malware Engine] [AVLoadFailure3] Cannot load engine McAfee Gateway Anti-Malware with index '573'. Reason: 'SCANMapiInitialize() failed with code: SCANMAPI_ERROR_LOADFAILED'.

[Anti-Malware Engine] [AviraCannotInitializeSavapi] Avira: Avira: Cannot initialize savapi3 due to error code 10.

 

[Anti-Malware Engine] [ErrorFromAVEngine] Error message from engine: 'McAfee Gateway Anti-Malware Engine: failed to load MFE base API in '/opt/mwg/plugin/data/antivirus/SCANM/225''

[Anti-Malware Engine] [ErrorLoadingSCANM] Failed to load and initialize 'McAfee Gateway Anti-Malware' engine from directory '/opt/mwg/plugin/data/antivirus/SCANM/225'. Error code: 'SCANMAPI_ERROR_LOADFAILED'.

[Anti-Malware Engine] [AVLoadFailure3] Cannot load engine McAfee Gateway Anti-Malware with index '225'. Reason: 'SCANMapiInitialize() failed with code: SCANMAPI_ERROR_LOADFAILED'.

 

After the first occurrence, we had to "service mwg restart" to restore services on that device.

 

During the second occurrence, the initial attempt to perform a service restart resulted in failure to stop the anti-malware engine.

 

We are configured with Block on Anti-Malware Engine errors in Error Handler.

 

Prior to the second occurrence, I configured a new Error Handler rule to send a notification if Error.ID equals 14001 -- criteria based on the existing Error Handler block rule for Anti-Malware Engine Overload.

 

This rule did not activate and we ended up with over 4 dozen help desk tickets related to the problem. Since nearly 2000 connections were denied, 4 dozen problem reports is probably fortunate from our standpoint.

 

I have now expanded the notification rule to this criteria:

 

Error.ID equals 14000 OR Error.ID equals 14001 OR Error.ID equals 851 OR Error.Message matches Broken*

 

Will this be sufficient to provide a timely notification if this issue recurs? Any thoughts on why the original notification rule didn't activate?

 

If not, I need to know so that I can configure a non-MWG monitoring solution to trigger if an anti-malware overload message is received.

  • eelsasser McAfee SME 842 posts since
    Mar 24, 2010
    Currently Being Moderated
    1. Feb 26, 2013 10:45 PM (in response to btlyric)
    Re: anti-malware overloaded

    I forget which version you have, but 7.3 added some additional codes that probably cover those conditions. My guess is they were added to cover your use-case.

     

    • Affected ruleset: Block on Anti-Malware Engine Errors, Action: Rule modified, Affected rule: Block on Internal Anti-Malware Engine Errors
      • In 7.2.0 the rule criteria was: Error.ID equals 14002
      • In 7.3.0 the rule criteria is: Error.ID greater than or equals 14002 AND Error.ID less than or equals 14050
      • Reason: New error codes were added for the Anti-Malware engine.

     

    https://community.mcafee.com/docs/DOC-3348

  • fschulte Apprentice 57 posts since
    Nov 16, 2011
    Currently Being Moderated
    2. Feb 27, 2013 11:00 AM (in response to btlyric)
    Re: anti-malware overloaded

    Hi Btlyric!

     

    You should open a support ticket and provide them a feedback.

    Depending on the situation you are experiencing, a restart of mwg-antimalware could resolve the situation.

     

    Ciao

    Felix

  • jbmartin6 Newcomer 17 posts since
    May 14, 2010
    Currently Being Moderated
    3. Feb 27, 2013 12:25 PM (in response to btlyric)
    Re: anti-malware overloaded

    Do you have core dumps enabled? If so take a look and see if mwg-antimalware is generating cores. (/opt/mwg/logs/debug/cores/) We are having the same experience, but I haven't seen any correlation with the errors you mention. But there are a slew of antimalware cores in the 20 minutes or so before mwg-antimalware becomes unresponsive. Development is looking into the feedbacks and cores we've provided, apparently there is some issue in mwg-antimalware. If you have cores open a support case, they helped us out in alleviating the issue by identifying a few URLs that were causing mwg-antimalware to core dump. We whitelisted those and haven't had a repeat since. We still see a handful of cores each day but haven't had mwg-antimalware become completely unresponsive.

  • jbmartin6 Newcomer 17 posts since
    May 14, 2010
    Currently Being Moderated
    6. Feb 28, 2013 7:46 AM (in response to btlyric)
    Re: anti-malware overloaded

    I don't have any useful ones for you, they were all intranet sites or else required logons to get to.

     

    One I can share is:

    hxxps://ssl.gstatic.com/analytics/20130129/web/analytics__ru.js

     

    This caused a malware detection and core dump every time I went to it.

  • al.johnson Newcomer 22 posts since
    Dec 16, 2010
    Currently Being Moderated
    7. Feb 28, 2013 11:47 AM (in response to btlyric)
    Re: anti-malware overloaded

    We've been experiencing similar problems.  The first problem occurred Sunday and we spent several hours trying to figure out what was going on.  Finally were able to narrow it down to a single (of 8) device and rebooted it after taking a feedback.

     

    We had two more devices start blocking on AM engine load failure.  Since we know what to do we rebooted and did not have a significant outage.  (With 40,000 users, any blip in Internet access is noticed rather quickly).

     

    Monday we updated the Block if Antimaleware Engine cannot be loaded rule to send an email to those of us who can do something about it.  We caught one error yesterday at noon in time to reboot the offending device before customer notifications came in.

     

    Watching the devices closely today I saw all of the Avira AM engines restart at t 12:17pm EST. 

     

    This was indicated by several notices in all the core-errors logs:

    [2013-02-28 12:17:22.597 -05:00] [AV] [AVError] Error in AntivirusFilter: 'Cannot filter because special update is performed.'.

     

    And in the antimaleware-errors logs I see this:

    [2013-02-28 12:17:22.497 -05:00] [Anti-Malware Engine] [ExitRestartAppInternalKill] Stopped 'McAfee Web Gateway Anti-Malware Engine version: 7.2.0.1.0 - build: 13253' after internal kill requested...

    [2013-02-28 12:17:29.556 -05:00] [Anti-Malware Engine] [TermSignalReceived] 'McAfee Web Gateway Anti-Malware Engine version: 7.2.0.1.0 - build: 13253' - child process exited (termsignal='9').

    [2013-02-28 12:17:29.578 -05:00] [Anti-Malware Engine] [RestartAppFrequentFailCountOK] 'McAfee Web Gateway Anti-Malware Engine version: 7.2.0.1.0 - build: 13253' - restarting...

    [2013-02-28 12:17:29.600 -05:00] [Anti-Malware Engine] [StartApp] Starting 'McAfee Web Gateway Anti-Malware Engine version: 7.2.0.1.0 - build: 13253' - 'No FIPS mode'.

     

    #Further review of the Dashboard charts I see #that each of the devices that have failed in the past week all show a gradual memory utilization by mwg-antimalware beginning around 1200 EST on 2/20.  When the memory gets to 4G the device fails.  This is only showing on the devices that we have had failures on, the other devices have a mostly flat line for memory usage in mwg-antimalware.

     

    My hope is that the new engine that just went out is a fix.  I've got a message into the support team for verification.

  • al.johnson Newcomer 22 posts since
    Dec 16, 2010
    Currently Being Moderated
    8. Feb 28, 2013 2:42 PM (in response to al.johnson)
    Re: anti-malware overloaded

    Good news update from the support team.  An official notice is coming out.

     

     

    Engineering has identified an issue with the MWG AV process which caused older AV engines not to get unloaded properly after an update occurred. This issue slowly caused memory usage to increase with every AV update until the AV process reached its limit of 4GB.  Engineering identified a solution for this issue which was implemented with the release of "Gateway DAT" 1644 and in addition, an AV engine restart was triggered to bring the AV engine to a clean start.   The release of “Gateway DAT” 1644 and AV engine restart is automated and does not require any further actions to implement. 

     

    To verify that your MWG DATs are updated as described above, please review the following in the UI: 

     

    Dashboard >> Alerts >> Appliance Status >> Gateway DATs 1644 or newer Dashboard >> Alerts >> Appliance Status >> DATs 7000 or newer

     

     

    To verify that the issue is resolved, we recommend that you check the appliance for the next 3 days once a day with the following command:

     

    /usr/sbin/lsof -p $(pgrep -n mwg-antimalware) | grep "antivirus/SCANM" | grep "(deleted)"

     

    If the command returns no results then the issue has been eliminated.  If the command returns any results or if you need any further assistance please execute a new feedback file (Troubleshooting >> Feedback) and provide it to Support.

  • bumjubeo Newcomer 1 posts since
    Feb 28, 2013
    Currently Being Moderated
    9. Feb 28, 2013 2:50 PM (in response to al.johnson)
    Re: anti-malware overloaded

    Good News!

     

    I'm going to submit a ticket either way we had 1 of our gateways crash out today as well.

1 2 Previous Next

More Like This

  • Retrieving data ...

Bookmarked By (0)

Legend

  • Correct Answers - 5 points
  • Helpful Answers - 3 points