1 2 Previous Next 10 Replies Latest reply on Dec 7, 2016 5:20 PM by Jon Scholten

    Don´t wait for ICAP Server response

    metalhead

      Hi,

       

      I have configured MWG 7.6.2 with the RESPMOD rule to successfully pass file downloads to a sandbox solution via ICAP.

      Now the MWG always waits for the answer from the ICAP server before passing the file to the user.

       

      Is it possible to switch this to "background detetction" - so that the MWG passes the file to the ICAP server and immediately delivers the content to the user ?

       

      I used the default RESPMOD rule from the "ICAP Client" ruleset in the library.


      Thanks Tom

        • 1. Re: Don´t wait for ICAP Server response

          It is possible to communicate with the ICAP server or run any other lengthy task in background.

           

          Have a look at "MATD - Offline Scanning With Immediate File Availability" ruleset in the library. Place your ICAP client ruleset at the top of your policy and set the first ruleset condition to "Antimalware.MATD.IsBackgroundScan is true". Add action "Stop cycle" at the end of the icap client ruleset. Add a rule that uses property Antimalware.MATD.InitBackgroundScan at the place in your policy where you've originally placed icap client ruleset.

           

          Policy before the change:

          <Ruleset 1>

          <Ruleset 2>

          <ICAP client: Conditions: condition1 and condition2>

          <Ruleset 3>

           

          Policy after the change:

          <ICAP client: New Condition: Antimalware.MATD.IsBackgroundScan is true>

                   <All rules from the original ruleset>

                   <New optional rule: notify administrator if file was blocked/infected/etc>

                   <New must have rule: condition: always, action: stop cycle>

          <Ruleset 1>

          <Ruleset 2>

          <New Ruleset: Conditions from original ICAP cleint ruleset: condition1 and condition2>

                   <Rule Init offline scan: condition: Antimalware.MATD.InitBackgroundScan is true, action: continue>

          <Ruleset 3>

           

           

          Andrej.

          1 of 1 people found this helpful
          • 2. Re: Don´t wait for ICAP Server response
            metalhead

            Hi Andrej,

             

            I tried around a bit. Everytime this rule is active I get the error below:

            Rule Init offline scan: condition: Antimalware.MATD.InitBackgroundScan is true, action: continue

             

             

            Internal Anti-Malware Engine Error 

            The Anti-Malware engine encountered an internal program error while handling your request and your administrator doesn't allow to deliver content without being checked for viruses. Please call your administrator with the error message below. 

            URL: http://172.27.254.10:8081/malfiles/0test/

            Error Message: (14013) Internal MATD filter error: Could not initiate background call in time.

             

             

            Remember I actually don´t use a McAfee ATD !

             

            Regards Thomas

            • 3. Re: Don´t wait for ICAP Server response

              Hi,

               

              you don't need to have ATD to use this properties.

               

              Property Antimalware.MATD.InitBackgroundScan creates a virtual copy of the current transaction that is handled by rule engine independent from the original transaction. You see an error message because you didn't implement second ruleset at the top of your policy that is activated by the condition "Antimalware.MATD.IsBackgroundScan equals true".

               

              Please have a look at the documentation or ATD ruleset in the local rule library and post a screenshot of your rulesets if it still doesn't work.

               

              Andrej.

              1 of 1 people found this helpful
              • 4. Re: Don´t wait for ICAP Server response
                Jon Scholten

                Hi Thomas,

                 

                I have an example ruleset of what guru Andrej is describing.

                 

                You import and use the default "Initialize background scan ruleset", this forces the separate background transaction. The background transaction is the virtual copy Andrej mentioned.

                 

                When MWG sees this background transaction, the property "IsBackgroundScan" will be true, so you can create a dedicated top level ruleset for it:

                 

                 

                You would then have REQUEST based rules inside of the ruleset which look similar to the ones below. The ones below show how you can add ICAP headers to the ICAP request, and also parse the ICAP response headers sent back by the ICAP server:

                 

                 

                Best Regards,

                Jon

                1 of 1 people found this helpful
                • 5. Re: Don´t wait for ICAP Server response
                  Jon Scholten

                  When I wrote this last post I was boarding an airplane and the second image never got posted.... Here is what the offline scan ICAP rules would look like:

                   

                  The rules include an example of how you can pass a header to the ICAP server (first rule), and also how you can parse a header that the ICAP server may pass back to MWG (last rule).

                   

                  I've also attached example rules. The example uses the MWG itself as the ICAP server.

                   

                  Best Regards,

                  Jon

                  1 of 1 people found this helpful
                  • 6. Re: Don´t wait for ICAP Server response
                    metalhead

                    thanks to all for your valuable input.

                    We got it working.

                     

                    Regards Tom

                    • 7. Re: Don´t wait for ICAP Server response
                      roybad

                      Apologies for replying to an oldish thread but firstly, really nice idea (although, I'm not sure it will work when we do have ATD in the mix)

                      but the question I have is -

                      In the example Jon wrote, it relies on the ICAP server returning a header of some sort. Great, fantastic - IF the ICAP server returns something meaningful in a header.

                       

                      My problem is the ICAP server we have doesn't return any headers but changes the content when it returns a 200 as oppose to a 204. All well and good but the return messages from the ICAP server are rubbish and not something we can make pretty with HTML or present to the end user.

                      We want to use some of MWG's logic to present a nice exception page with the returned message from the ICAP server.

                       

                      Only problem, there's no way I can see to grab the returned Body from the ICAP server on MWG. of course, if the returned content is a rewritten page, we don't want to interfere.

                       

                      The only way I found was to forward the HTTP traffic to a next hop proxy http listener on the same proxy instead of calling ICAP. Then, from the "new HTTP/s request", have the proxy call ICAP there. The ICAP will return a body (that this cycle can't interrogate) and we pass it back to the same proxy (returned from nexthop) which can then process it.

                       

                      It does mean we'd have to break SSL twice for the same content and pass any specific headers back and forth.

                      Seriously painful and adding steps and latency we don't need - and all we want to do is interrogate the content that comes after the HTTP/1.1 200 OK (e.g. Site blocked as you put some dodgy content there)

                       

                      Anyone have any ideas or is the passing traffic back to the same proxy the only way of looking at the body that ICAP returns? Headers seem fine but not the body. There isn't even a Response Cycle when ICAP returns a 200 from a ReqMod call.

                       

                      Any help would be really nice,

                       

                      thanks

                      • 8. Re: Don´t wait for ICAP Server response
                        asabban

                        Hi Roy,

                         

                        I think I had a similar issue at a customer where the ICAP response should be manipulated. I tried several things but ended up with a similar solution to what you described, but rather than a next-hop proxy call we did another ICAP call, e.g. MWG handed the file to itself via ICAP on a separate port, in this ICAP call the file was sent to another ICAP server, then eventually changes were applied to the body and returned to the initial execution of the rule engine.

                         

                        In this setup MWG is running as a proxy in front of an upload portal, so the additional execution of the rule engine is not a problem. I won't recommend this in a forward proxy environment where every request is treated like this.

                         

                        I am not sure if there is a way to improve your setup. I think it would be helpful to better understand the problem if we had some examples, a packet capture and a sample how the communication looks and how it should look after processing. If you're interested I am happy to have a look and work on this with you, however I am not able to promise a solution right now.

                         

                        Best,

                        Andre

                        • 9. Re: Don´t wait for ICAP Server response
                          roybad

                          Thanks Andre

                          Thanks for the mail and help.

                          We tried the icap->icap chain using MWG as an icap server but we can't interrogate the body on either step of the icap request - What's with MWG and making that response unavailable.

                          How did you get it working ?

                          I'm going to run some load testing with the multi loops but icap is a real pain in the neck

                           

                          Thanks!!!

                          1 2 Previous Next