1 of 1 people found this helpful
I believe you need to create a web policy mapping based on the user agent string.
Add a policy mapping layer and position it before any existing policy mappings - edit the mapping and enter the following information:-
- Extract user information from = Extract defined request header
- User defined meta or request header = User-Agent
- Enable shell expressions in mapping rules = checked/ticked
Then, assign a policy based on the trigger criteria ~Akamai*
In our case we have a policy on the Web Gateway called "Open" and this policy has very little enabled (certainly none of the anti-malware options) because the Akamai downloader doesn't seem to like having anything interfere with the data while it is being downloaded. The use of this mapping allows the "Open" policy to be used based on the client application (the Akamai download manager, in this case), rather than a policy based on the user name, group name, or IP address.
Hope this helps.
before creating a mapping from user-agent which might screw up your reporting, it is advisable to understand why something is not loaded as expected. Do you have an example of content you are trying to get from the MS pages?
You can also turn on Filter Tracing to see if the request is blocked by any filter.
Additionally it is noteable that some download managers require special prerequisites in order to work correctly, such as taking the content length header to estimate progress and time, etc. This header for example is something that MWG strips as there might be situation where the content is modified and the header is not valid anymore. Usually the MS download servers are covered by internal components of the product, but it is advisable, as said, to understand the root cause of this to make it go away with good results.
From my exprerience with the downloader that Microsoft uses, it uses partial downloading, meaning instead of downloading all of the file at once, it will requests parts of the download (see 10.2.7 206 Partial Content). This allows you to download some of the file, pause it, and continue later. The nature of this is inherently not very secure as it doesn't allow any virus scanner to scan the entire file, so by default this is not allowed on the Web Gateway.
To allow for this behavior you can go under Proxies > ICAP(S) Server > REQMOD Settings, then there is an option for 'Forbid Partial Downloads (HTTP)', unchecking this box will allow for '206 Partial Content' responses. The help file contains additional information on this as well.
Alternativley you can ICAP bypass for the site, which would bypass filtering but would only allow this for that particular site.
It is noteable, though, that allowing partial downloads can drill a whole into your security infrastructure. A file looks like ABCDEF12345GHI, from which 12345 is a virus partial download can make the file appear in a diffrent order such as 45GHI 23ABC DEF1, which will only reassamble the virus on the client. Additionally if ACDE are the magic bytes for a file type all other parts will appeat as binary garbage to the gateway, there for they can also by pass the media type filter.
Thanks for the replies.
I've tried capturing the traffic on the Gateway using a tcpdump / wireshark but couldn't see anything being denied.
I also tried on our Dev gateway, allowing the partial downloads. That allowed the download manager to start but then stopped at various points. 65%, 80% and 95%.
This was stopped by heurestic scanning.
GET http://akamdub.fullproduct.download.microsoft.com/dl/download/release/7/3/5/SW_DVD5_Win_Pro_7_32BIT_English_Upg_MLF_X15-73572.ISO?LCID=1033%26PGM=VLSC%26TID=22140076%26__gda__=1270652325_d9e85a1b1f5aa0e63d0b948e506b6737 HTTP/1.1" 403 1051551 2266 "" "AkamaiDLM/220.127.116.11 ActiveX (<OS:Windows XP Professional>-<Browser:7.0.5730.13>)" "" 80 "application/x-iso-9660" "SL-Webadmin" 9.014 "McAfeeGW: Heuristic.BehavesLike.Exploit.CodeExec.PGPG" Trusted - "HTTP Response Header Filter" "Object infected: McAfeeGW: Heuristic.BehavesLike.Exploit.CodeExec.PGPG." This is despite the fact that I whitelisted akamdub.fullproduct.download.microsoft.com from any scanning at all.
OK I think I found the way forward.
I added a whitelist entry for *.microsoft.com/*.iso
I am just fine-tuning the entries at the moment.
I tried this with the "Forbid partial downloads" box unchecked. However this is a global setting.
I have blocked all .exe downloads at the moment and whitelisted certain sites from the Media Type Filter for Adobe, etc.
I've now checked the box again and have excluded *.microsoft.com from the HTTP reqmod settings. That works too.
My question way, which is the greater risk to security ?
Allowing partial downloads, with the proviso that only whitelisted sites can download exe files, or adding exceptions to the ICAP services ?
Thanks for your help with this.
Tough to answer, as the security implication lies in the eye of the beholder, so it is up to you to decide which way to walk. I can try to outline what the effects of each decision might be.
A global ICAP bypass will bypass all filters for the entered criteria - it will remain 100% unscanned - as benefit none of the traffic's characteristics will be changed. So you should be very precise in defining the criteria.
Allowing partial downloads on a global basis, might mitigate the nasty side effects of downloaders, which try to do partial file transfers, but also will drill the aforementioned hole in your security setup.
I'd like to suggest a 3rd possible method as possible solution (didn't test myself - sorry).
create a config backup over the UI.
ssh to the applicance as root.
- cd /opt/webwasher-csm/conf
- vi global.conf
- in vi search for Special (just type in /Special)
use the cursor keys and :
- scroll to the line " SpecialUpdateServer='*.windowsupdate.com$;windowsupdate.microsoft.com$;w2ksp*.mi crosoft.com$;office.microsoft.com$;download.microsoft.com$;update.microsoft.com$ ' "
- navigate to ...download.microsoft.com$.... and change the entry to *.download.microsoft.com$ as this matches the akamei URL (to change, press i when at the position)
- now save the file (ESC, followed by :wq!)
check if the file belongs to wwasher:wwasher (ls -la global.conf, should similar to -rw-r--r-- 1 wwasher wwasher 74013 Apr 10 00:47 /opt/webwasher-csm/conf/global.conf)
- IF user and group are incorrect do chown global.conf wwasher:wwasher
- The bad news is, you will have to restart Webwasher : service webwasher-csm restart
Check if it works now. In case it doesn't there is no need to change the setting back.
can you tell us a little bit more about that SpecialUpdateServer section in the global.conf does?