I am a bit pussled about the time WebGateway takes to scan complex files. I do know that archives can contain a _lot_ of small files for which the ruleset is executed each time and yes, I am aware of AvOptimization rule sets.
I am currently looking at a progress page which indicates that it has already taken more than 2 hours to scan a ~170 MiB installer package. What seems strange to me is that there is just ~10% CPU-usage and 0.2 load on that proxy according to top.
So why is this taking so long?? Cant be the CPU or hard-drive being too slow ...
top - 16:14:30 up 30 days, 7:35, 1 user, load average: 0.20, 0.25, 0.28
Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
Cpu0 : 0.3%us, 0.0%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 5.3%us, 0.0%sy, 0.0%ni, 94.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 2.3%us, 0.3%sy, 0.0%ni, 97.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 3.7%us, 0.0%sy, 0.0%ni, 96.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12197776k total, 9512340k used, 2685436k free, 875560k buffers
Swap: 16777212k total, 0k used, 16777212k free, 4169664k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8208 mwg 20 0 1625m 1.5g 41m S 9.3 12.8 123:24.08 mwg-antimalware
8300 mwg 20 0 2404m 1.3g 89m S 1.7 11.2 518:30.16 mwg-core
Hope someone can enlighten me.
Thanks, but I am already aware of how the ruleset affects scan time and how to mitigate that.
The point of my question is why I don't see any noteworthy CPU load during that time. I would expect to see at least one core at 100%.
Oh, and BTW the scan fails completely when using layered inspection, this is McAfee only.
We get that too from time to time on MEG as well (low CPU/load on appliance while it takes forever to scan)
MEG or MWG can take minutes/hours to scan something and once scannd, if you pass it through a VSE Scan, it takes seconds only.
So far, the only answer we have received from McAfee support was to implement a timeout function (MEG already has 8 minutes of scan time max by default), which is basically just a band-aid.
A timeout function? So basically, I care more about delivering that virus to the user faster? Doesn't sound like my security policy should be altered just by how long it takes to scan a file. Or what else is the timeout doing, other than aborting the scan? How have other users tweaked the configuration (other than by using the AV optimization rule set)?
I feel similar about this. But since users will just divert to other means of getting those files I can't leave it that slow.
What I did for now is to limit the opener to just two levels once the AV-timeout hits. That makes it much faster.Nachricht geändert durch malte.schroeder on 15.07.14 06:30:30 CDT
from a security perspective it might sound weird to use any kind of timeout or exception to "skip" AV and send the file to the user. However I learned in the recent past that your users actually don't care that much about security than you probably do. They are waiting for their fiel to continue work (or probably just click through a PDF full of funny cat images).
So what they experience is clicking a link and while at home the file is immediately (or after a short time) presented at work they need to work for that "damn security check". Users want their files quickly. If the file is not delivered quickly enough they start the download again and again or complain. Based on that I have seen multiple requests from customers who want to change from a strict security policy to a security policy that applies as much security as possible, but without impacting the users wherever possible.
A similar example is the usage of social websites, such as facebook. A few years ago almost everyone blocked such categories, because such websites were not "work related", but meanwhile customers allow access but ask for ways to "secure" such websites by making them read-only or block specific features.
So yes, from a security perspective it does not make sense to make exceptions. But users expect their files to arrive and they do not want to wait an hour for a scan to finish, so MWG scans as much as possible and delivers the file. The desktop AV is the next one in the long chain of security products that looks at what is done during exctration/installation, and will stop a virus from there.
We even have a feature request (I am not sure if it was filed officially but I was asked it this can be made possible) to have a "Skip Filtering" button on the progress page to allow users to abort the scan whenever they feel receiving the file is more important than security. That is basically why we have options to "skip" AV filtering based on time or number of objects. For me it sounds strange as well, but it seems to be a trend to no longer annoy users with security 😉
is also see this behavior on some files. Perhaps i can post a link here. The normal Download of the file, when the URL was whitelisted in the Rulese, takes about 10 minutes. When downloading and scanning the fie, using Data Trickling, we waited more than 40 minutes. But we received no file. Some times, when using a donwload page, you can see a quick download, but the scan does more than an hour.
This also happens sometimes with the latest version on MWG.
I drink to Andre's post.
This is EXACTLY what we're facing day-to-day with the different security products/policies we manage.
Users don't mind security, for as long as it makes their life easy. The minute you start making it a bit harder for them to do what they want to do, you get push-backs.
At that point it's a matter of company philosophy. When users complain about security hindering their ability to do their job efficiently, the solution in management's mind is easy: remove security.
In the end, you have to learn to compromise. We implemented a 'no-scan' for big files from sites with good reputation in order to avoid these issues. We simply figure that once the file is downloaded it gets scanned at the endpoint anyway. Also, chances that a big file from a trustworthy site is malicious are rather low.
Oh, and if you think the scanning at the gateway issue only exists with McAfee, I can tell you first hand that it's not the case. Might not be with the same files but all gateway products we've tested had caveats when it came time to scan archive files with large numbers of small files inside.
i fully agree with you. We also built such policies for customers e.g. not scanning specific file types and file sizes from "secure" hosts.
But in fact, it could be useful to know whats going on with MWG. Perhaps anything is going wrong. Perhaps any specific necessary checks.
If there are important reason, this can be explained to the customer implementing a necessary security level.
For me, such information is absolutely important as a base for security discussions.