ICAP Example Help

[numark]

Hello,

I am looking for some help setting up a new ICAP client that will send file requests to MWG.

RESPMOD icap://10.x.x.x/avscan ICAP/1.0

Host: 10.x.x.x

User-Agent: IT-Kartellet ICAP Client/1.1

Allow: 204

Preview: 30

Encapsulated: req-hdr=0, res-hdr=1, res-body=22

I am trying to figure out what sets the res-hdr and res-body number. Currently I have these statically assigned but I feel like we should some how have these dynamically generated based on the file we are processing.

Any help is appreciated! Thank you!

[eelsasser · Accepted solution]

An ICAP request consists of a few sections:

The ICAP Request itself, with various Headers. This would look like this:

RESPMOD icap://192.168.2.231:1344/RESPMOD ICAP/1.0\r\n

Allow: 204\r\n

Connection: close\r\n

Host: 192.168.2.231\r\n

X-Client-IP: 192.168.1.2\r\n

Encapsulated: req-hdr=0, res-hdr=84, res-body=150\r\n

\r\n

The Encapsulated Request Header. This is supposed to represent an HTTP request.

GET /testfile.zip HTTP/1.1\r\n

Host: 192.168.1.2\r\n

\r\n

The Encapsulated response header.

HTTP/1.1 200 OK\r\n

Transfer-Encoding: chunked\r\n

Content-Length: 0\r\n

\r\n

And the Response body, which is where the file is and it's usually chunked binary.

211\r\n

...........\r\n

(529 bytes of binary data sent)\r\n

...........\r\n

0\r\n

\r\n

req-hdr=0 means the "GET /eicar.com HTTP/1.1"   starts at 0 bytes right after the ICAP header.

res-hdr=84 means the HTTP/1.1 200 OK starts at 84 bytes after the ICAP header.

res-body=150 means the Response body starts at 150 bytes after the ICAP header.

Does that help?

I attached an ICAP client PERL script that _might_ make it easier to follow.

[numark]

Thanks Erik for your quick reply. This information has been extremely helpful for me.

Thank you!

[jebeling · Accepted solution]

The attached code is for test and example purposes only, it is not production quality, and it is not supported in any way.

Attached is a heavily modified version of Erik Elsasser's original java based ICAP test client. Also attached is an example ICAP server ruleset that can be used in conjunction with this client or others.

The code could be much cleaner, but seems to be fully functional in limited testing. Suggestions and modifications welcome.

jar file can be found in ICAPTester_5_1\dist

Simply double clicking on the jar file will run it in interactive mode if JRE is on the system.

 

Inputs are not fully validated in either operational mode!

Command line currently expects 0-5 arguments:

java -jar ICAPTester_5_1.jar <serveraddress[: portnumber]> <service> <filepath or url> <allow204> <preview: previewbytes>

<serveraddress[: portnumber]> = sever address of ICAP server with optional port number for ICAP service, if no port is specified 1344 is used

<service> = wwreqmod (icap REQMOD), wwrespmod (icap RESPMOD),  avscan (avscan is equivalent to wwrespmod), interactive, options, gtiget, or fetchurl

<filepath or url> = full file path for wwrespmod or avscan, full url for wwreqmod

<allow204> = any string other than "no204" results in allow 204

<preview: previewbytes> = string beginning with preview turns on preview, number of bytes to send indicated by numerical string following preview

Command line operation currently defaults to allow 204, no preview

Command line operation with more than 5 arguments results in usage error and launches in interactive mode

Command line with no arguments launches in interactive mode

Command line examples with output (ICAP Server at 192.168.11.122):

java -jar ICAPTester_5_1.jar 192.168.11.122 wwreqmod "http://www.mcafee.com" no204 preview:30 foo

Running with Options: 192.168.11.122 wwreqmod http://www.mcafee.com no204 preview:30 foo

ClientIP:foo

Invalid Options: 192.168.11.122 wwreqmod http://www.mcafee.com no204 preview:30 foo

Usage: ICAPTester serverName<:serverPort> <serviceName>  <filepath> <no204> <preview:#>

Defaults:         192.168.11.122:1344     avscan  allow204 nopreview  C:\Temp\putty.exe

Entering interactive mode

java -jar ICAPTester_5_1.jar 192.168.11.122 wwreqmod "http://www.gambling.com"

Running with Options: 192.168.11.122 wwreqmod http://www.gambling.com

ClientIP:192.168.197.1

Result string from ICAPClient:

Status: Blocked:

BlockResult: 403

Virus: "Unknown"

Categories: "Gambling"

Reputation: 0

Geolocation: US

java -jar ICAPTester_5_1.jar 192.168.11.122:1344 wwrespmod "C:\users\user1\Documents\test.html"

Running with Options: 192.168.11.122:1344 wwrespmod C:\users\user1\Documents\test.html

ClientIP:192.168.197.1

stdOut: Debug ICAPClient.scan sending file: C:\users\user1\Documents\test.html

Result string from ICAPClient:

Status: Allowed: No mod needed:

BlockResult: 204

Virus: "No malware found"

Categories: "No X-Categories"

Reputation: 15

Geolocation:

java -jar ICAPTester_5_1.jar 192.168.11.122 wwreqmod "http://www.mcafee.com/index.html?arg=99"

Running with Options: 192.168.11.122 wwreqmod http://www.mcafee.com/index.html?arg=99

ClientIP:192.168.197.1

Result string from ICAPClient:

Status: Allowed: No mod needed:

BlockResult: 204

Virus: "No malware found"

Categories: "Business, Software/Hardware"

Reputation: -15

Geolocation: US

java -jar ICAPTester_5_1.jar 192.168.11.122 avscan "C:\users\user1\Downloads\eicar.com" no204

Running with Options: 192.168.11.122:1344 wwrespmod c:\users\user1\documents\test.html no204

ClientIP:192.168.197.1

Result string from ICAPClient:

Status: Blocked:

BlockResult: 403

Virus: "Virus Found: EICAR test file"

Categories: "No X-Categories"

Reputation: 15

Geolocation:

java -jar ICAPTester_5_1.jar 192.168.11.122:1344 options

Running with Options: 192.168.11.122:1344 options

ClientIP:192.168.197.1

Result string from ICAPClient:

OPTIONS icap://192.168.11.122:1344/options?profile=default ICAP/1.0

Host: 192.168.11.122

ICAP/1.0 200 OK

Methods: REQMOD, RESPMOD

Options-TTL: 3600

Encapsulated: null-body=0

Max-Connections: 400

Preview: 30

Service: McAfee Web Gateway 7.8.0 build 24353

ISTag: "00000951-14.42.54-00008767"

Allow: 204

java -jar ICAPTester_5_1.jar 192.168.11.122 getgti "c:\users\user1\documents\gtiGetTestURL.txt"

URL,Categories,Reputation,Geolocation

www.eicar.org,"Information Security",2,DE 

www.mcafee.com,"Business, Software/Hardware",-15,US

http://www.gambling.com,"Gambling",0,US

www.google.com,"Search Engines",0,US

NOTE: getgti option expects text file with one URL or hostname per line

java -jar ICAPTester_5_1.jar 192.168.11.122 fetchurl "http://eicar.org/download/eicar.com"

Running with Options: 192.168.11.122 fetchurl http://www.eicar.org/download/eicar.com

ClientIP:192.168.11.77

Result string from ICAPClient:

   Status: Blocked:

   BlockResult: 403

   Virus: "Virus Found: EICAR test file"

   Categories: "Information Security"

   Reputation: 2

   Geolocation: DE

[jebeling]

In order to make the getgti command line option to work properly in the example code, you need a modified version of the ICAP Server Ruleset to insert the GTI data in X headers. The ruleset is attached.

Rule Sets

ICAP Server [✔] Enabled [✘] Disabled in Cloud Applies to: [✔] Requests [✔] Responses [✔] Embedded Objects 1: Connection.Protocol equals "ICAP" Enabled Rule Action Events Comments [✔] Enabled Fix Blank Host - Layer 7 1: URL.Host equals "" Continue Set URL.Host = "localhost"   [✘] Disabled X-Client-IP 1: Client.IP is in range 192.168.2.0/24 2: OR String.ToIP(Header.ICAP.Request.Get("X-Client-IP")) is in range 192.168.2.0/24 Continue   Example of how to use X-Client-IP: header. [✘] Disabled X-Authenticated-Groups 1: Authentication.UserGroups contains at least one match *Domain Admins* 2: OR String.Base64DecodeAsText(Header.ICAP.Request.Get("X-Authenticated-Groups")) matches *Domain Admins* Continue   Example of how to use X-Authenticated-Groups: header. [✘] Disabled X-Authenticated-User 1: Authentication.UserName equals "user" 2: OR String.Base64DecodeAsText(Header.ICAP.Request.Get("X-Authenticated-User")) equals "Local://user" Continue   Example of how to use the X-Authenticated-User: header. [✔] Enabled Report Categories X-Categories 1: URL.Categories<Default> does not equal Empty Category List° Continue Header.ICAP.Response.Add("X-Categories",List.OfCategory.ToString(URL.Categories<Default>)) Block if URL is in a malicious category. [✔] Enabled Report Categories Reputation X-Reputation 1: URL.Reputation<Default> does not equal 128 Continue Header.ICAP.Response.Add("X-Reputation",Number.ToString(URL.Reputation<Default>))   [✔] Enabled Report Geolocation X-Geolocation 1: URL.Geolocation<CloudOnly> is in list Geolocation: Country List Continue Set User-Defined.Geolocation = URL.Geolocation<CloudOnly> Header.ICAP.Response.Add("X-Geolocation",User-Defined.Geolocation) Lookup country the URL resides in, in case you want to block by country code. [✔] Enabled Enable Composite Opener Always Continue Composite Opener<Default> Opens the documents for scanning. [✔] Enabled MediaType: Detect 1: List.OfMediaType.IsEmpty(MediaType.EnsuredTypes) equals false Continue Header.ICAP.Response.Add("X-Media-Type",List.OfMediaType.ToString(MediaType.EnsuredTypes,", ")) Validate the actual media type by doing magic byte checking. [✘] Disabled MediaType: Block Not Detected 1: List.OfMediaType.IsEmpty(MediaType.EnsuredTypes) equals true Block<Media Type (not detected)> Statistics.Counter.Increment("BlockedByMediaFilter",1)<Default> Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Block if not in list of known media types. [✘] Disabled MediaType: Blocked Downloads 1: MediaType.EnsuredTypes at least one in list MediaType: Blocked Downloads Block<Media Type (Block List)> Statistics.Counter.Increment("BlockedByMediaFilter",1)<Default> Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) If media type is in given list during download, the user will be blocked. [✘] Disabled MediaType: Block Encrypted 1: Body.IsEncryptedObject equals true Block<Media Type (Not Supported Archive)> Statistics.Counter.Increment("BlockedByMediaFilter",1)<Default> Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Block if file is password protected. [✔] Enabled MediaType: Block Multipart Archive 1: Body.IsMultiPartObject equals true Block<Media Type (Multipart Archive)> Statistics.Counter.Increment("BlockedByMediaFilter",1)<Default> Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Block if file is a multi-part archive. [✘] Disabled MediaType: Block Corrupted Archive 1: Body.IsCorruptedObject equals true Block<Media Type (common)> Statistics.Counter.Increment("BlockedByMediaFilter",1)<Default> Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Block if file is corrupted and cannot be opened. [✘] Disabled Test Path Modification 1: Cycle.Name equals "Request" Continue Set URL.Path =      "/pathmodified" +      URL.Path   [✘] Disabled Test Redirect 1: Cycle.Name equals "Request" Redirect<Default> Set Redirect.URL = "http://www.mcafee.com"   [✘] Disabled Test Body Modification Beginning Always Continue Body.Insert(0,"Body Modified")   [✘] Disabled Test Body Modification Ending Always Continue Body.Insert(Body.Size,"Body Modified")   [✘] Disabled Test Body Modification 200 bytes in 1: Body.Size greater than 220 Continue Body.Replace(200,14,"Body Modified!")   [✔] Enabled URL Filter: ICAP Setting 1: URL.Categories<Default> at least one in list ICAP: Blocked Categories Block<URL Blocked> Statistics.Counter.Increment("BlockedByURLFilter",1)<Default> Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Block, if a virus was found in a response or embedded object [✔] Enabled Anti-Malware: ICAP Setting 1: Antimalware.Infected<Gateway Anti-Malware: ICAP Setting> equals true Block<Virus Found> Statistics.Counter.Increment("BlockedByAntiMalware",1)<Default> Header.ICAP.Response.Add("X-Virus-Name",List.OfString.ToString(Antimalware.VirusNames<Gateway Anti-Malware: ICAP Setting>,", ")) Header.ICAP.Response.Add("X-WWBlockResult",Number.ToString(Block.ID)) Header.ICAP.Response.Add("X-Block-Reason",Block.Reason) Block, if a virus was found in a response or embedded object [✔] Enabled Anti-Malware: Scan Completed Always Continue Set User-Defined.Body.Modified = Body.Modified Set User-Defined.Antimalware.Scanned = true Validate that Antimalware scanning occured for logs. If it gets to here, it passed the Antimalware rules and is clean. Body.Modified indicates if a page was cleaned of mobile code. [✔] Enabled Stop Cycle Always Stop Cycle   No further processing.

---

[Tom2]

Does this support ICAPS? 

In my testing I can only get it to work using ICAP and not ICAPS. 

[jebeling]

MWG supports ICAPS. The example test client can support fetching via HTTPS but the test client does not currently support ICAPS

[Tom2]

Maybe I need to take another look at it, as soon as I enable ICAPS I don't get a valid response.

I've added the root that signed the ICAP cert to every java keystore I could find unless I'm missing something else?

[jebeling]

Apologies. Thought I had responded to this. Thought you were asking about whether or not MWG supported ICAPS. The test client supports HTTPS for retrieving content. Unencrypted ICAP is hard coded into the java of the test client. You are welcome to modify but unfortunately I don't have the time to do that. Sorry.