cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted

Web Gateway takes longer time to Scan files when downloaded

Hi Guys,

I am facing longer scanning time by WG for few files for example apple.com/itunes/download.

Is the scanning duration directly proportional to the file size ? or it depends on the file format or way it has been packed?

when I tried downloading itunes from the above, it takes me almost more than 200s to scan after it downloads thru progress page.

Cheers

Srini

27 Replies
Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hi Srini,

the scanning duration cannot be related to the file size. There are 500 MB files which pass in seconds and 1 MB files that take 15 minutes 🙂

You are right, it is depending on how a file is packed and which types of files an archive (or self-extracting archive) contains. MWG takes files and extract them to the maximum level possible, or configured. So if there is an archive, which contains an archive, which contains more archive, which contains more data that we can extract (like a PDF), all of these files will be extracted and filtered. This is a huge difference to what a desktop AV does, so scanning times cannot be compared.

I have seen an MSI installer package which was 2 MB in size, but when extracting it it contained more than 9000 small files. Extracting and filtering 9000 small files certainly takes some time, so it took longer than you would expect, and also longer than the desktop AV which just looks at the 2 MB file.

So I would say iTunes is not really a very simple file, but contains a lot of data and a lot of files which we can extract and which we apply filters to. So 200 seconds should be ok.

Best,

Andre

Highlighted

Re: Web Gateway takes longer time to Scan files when downloaded

Thanks Andre  for your valuable suggestion.

Highlighted
Level 9
Report Inappropriate Content
Message 4 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Is there any way to log what is happening with antimalware scanning module? ... I've been experiencing same problems while e.g. downloading Firefox 13.0.1.exe from Mozilla site. It takes more than 1000 seconds to scan that executable (it has ~65 files inside of it). Device (WG 4000) CPU usage is less than 30% at that time.

Thanks, MSM

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 5 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hello,

there is no debug log that allows to "look into the engine". It is possible to add a log event to the composite opener rules and write down whenever an object has been extracted, but this will only tell when MWG has extracted something and does not indicate what is happening in the engine.

If the issue is reproducible it would be great if you can share the complete URL of the file you download. We can check with a default MWG and see if we can replicate the issue.

Best,

Andre

Highlighted
Level 9
Report Inappropriate Content
Message 6 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hello Andre,

Here is an example:

Date: 24.7.2012

Link: http://download.cdn.mozilla.net/pub/mozilla.org/metrics/14.0.1-funnelcake14/win32/en-US/Firefox Setu...

Scanning time: 1100 seconds

Thanks, MSM

Highlighted
Level 12
Report Inappropriate Content
Message 7 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

One idea I'd throw out there is that you might consider building a trusted vendors URL list and consider it for bypass of the opener and anti-malware portions.  Be careful of course of forums though to treat them a bit separately.     The scan times I've seen as we've fully loaded some MWG's here lately have been eye poppingly awful, so we'll have to tune more of these.    The opener level of 100 seems like one hell of a generous default, and as others have said, reducing it to 5 seems to make a difference.  Finding the middle ground we can live with will be iterative.

Also, what are folks doing on a max scan size?  When we were doing Bluecoat proxies at this client, a max file size to scan was 50MB.   How can you specify a max size on the web gateways?    Skip antimalware if response cycle Body.size > X kB?  

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 8 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hello,

I have downloaded the file you have mentioned above through my MWG test machine. It is a VM with 2 GB of memory and 1 CPU. The filtering took around 15 seconds.

So I think 1100 seconds are a little too long! Which version are you running on? Did you try the download with the latest 7.2 release?

Best,

Andre

Highlighted
Level 9
Report Inappropriate Content
Message 9 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hello Andre

We are running Server version 7, UI Version 7.1.5.1.0 (11447), but we have no planned changes for that equipment during Q3 and Q4. CPU usage is less than 30% during scanning operation. We can accept that we made errors in equipment configuration, but as this is really basic functionality that in our opinion shouldn't be dependent on software version, we can't accept need for upgrade.

Thanks, MSM

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 10 of 28

Re: Web Gateway takes longer time to Scan files when downloaded

Hello,

Firefox is using the 7z algorithm to compress the installer. The 7z support on MWG had some glitches in the past, causing objects to take very long. I just installed a 7.1.5.1.0 build 11447 with the default configuration and I was able to replicate the issue you see, as downloading firefox took very long (I stopped after 5 minutes).

I updated this machine to the latest version and the download was completed within a couple of seconds as mentioned earlier. The only recommendation I have is to schedule an update, since this seems to solve the problem.  If you have a VM or physical test machine which does not serve users you could upgrade the box and check against your configuration, I am pretty sure that the issue will vanish.

Besides whitelisting I do not see a proper way to solve the problem on 7.1.5.1.0, sorry. You could try talking to technical support to get some additional advice.

Best,

Andre

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community