1 2 Previous Next 14 Replies Latest reply on Feb 23, 2012 7:37 AM by iain.gardiner

    Google.com/.co.uk being blocked - any ideas?

    iain.gardiner

      google_error.bmpI have a problem with google. With the common rule (enable opener) and the HTML filter (HTML opener) enabled google is blocked. If I disable either rule it works. If I disable the block rule on on corrupted archives only it also works. These rule are straight from the library without any customization yet.

       

      Looks like a conflict with the 2 openers.

       

      common rules.bmphtml opener.bmp

       

      Message was edited by: iain.gardiner added blocked message on 21/02/12 09:12:14 CST
        • 1. Re: Google.com/.co.uk being blocked - any ideas?
          asabban

          Hello,

           

          I have imported both default rules into my MWG, but I cannot replicate the problem. Certainly with the two openers activated there are a lot of additional objects going through the rule engine, I believe that the combination of both sends an object through the rule engine you do not expect on a first view.

           

          The block message looks like the media type filter has blocked the object. Maybe you can extend the error template to also log the triggered rule and maybe the filename or some other properties that will allow to better understand which object is causing this behaviour?

           

          Best,

          Andre

          • 2. Re: Google.com/.co.uk being blocked - any ideas?

            I had this exact same problem starting today.  Yesterday, I upgraded to version 7.1.6.1.0 (Build 12742).   Google is not the only website that i received this particular error.  Another is:  www.cnbc.com

            • 3. Re: Google.com/.co.uk being blocked - any ideas?
              iain.gardiner

              Hi Andre, log shows the following:

              [21/Feb/2012:16:43:24 +0000]" " "403" "BlockReason: Media type blocked" "URL Categories: Search Engines" "Blocking Rule: Block Corrupted MediaTypes" "GET http://www.google.com/ HTTP/1.1" "22" "URL Reputation: Minimal Risk" "Media Type: text/html"0 "User Agent: " "

              [22/Feb/2012:08:20:43 +0000]" "403" "BlockReason: Media type blocked" "URL Categories: Finance/Banking, General News" "Blocking Rule: Block Corrupted MediaTypes" "GET http://www.cnbc.com/ HTTP/1.1" "22" "URL Reputation: Minimal Risk" "Media Type: text/html"0 "User Agent: " "

               

              I'm running on version 7.1.0.6.0 build 12651

              • 4. Re: Google.com/.co.uk being blocked - any ideas?
                asabban

                Hello,

                 

                it looks like you have a rule that blocks files that are detected as corrupted. This rule is inserted automatically depending on what you have chosen from the initial installation wizard. It seems that one of the extracted objects (extracted by the opener) is detected as corruped incorrectly (or maybe it is corruped) and therefore the whole request is blocked. Either this is caused by a wrong detection or by an unlucky combination of rules.

                 

                I believe we should review the complete policy to find out how the rules interfere each other. For the moment I would recommend to disable blocking corrupted Media Types, if that is possible for you. Please provide a feedback (can be created from within the Troubleshooting tab) and file an SR. Someone from support should pick up the feedback and replicate the issue and find out which object is exactly blocked and why.

                 

                Would that be working for you?

                 

                Best,

                Andre

                • 5. Re: Google.com/.co.uk being blocked - any ideas?
                  iain.gardiner

                  Hi Andre, as the block only happens when both content opener + html opener are envoked it does seem like a bug. It doesn't think there is a corrupted archive if the HTML opener is disabled.

                   

                  I'll raise a ticker as see what engineering have to say. Interesting it's not just me that's having this issue.....

                   

                  Cheers

                  • 6. Re: Google.com/.co.uk being blocked - any ideas?
                    asabban

                    Hi Iain,

                     

                    it is possible that both openers are required to extract an object which is detected as broken. For example the composite opener could extract something from the body, which is usually not being marked as corrupted. This (extracted) object is then further extracted by the HTML opener, which extracts an object that is broken, so a piece of information that has been extracted from another piece of information that has been extracted from the original body.

                     

                    If you disable one of the openers this won´t occur any longer, e.g. the composite opener extracts only valid objects, and the composite opener is the one that makes the object that the HTML opener extracts available. So if one of both is turned off the issue won´t show up.

                     

                    As far as I know the rules that block corrupted objects come from one of the default policies (depends on how you initially set the sliders and region/enterprise). Do in case you used a common default policy from the initial wizard + adding the HTML opener can make the issue occur. This will also explain why other customers can see the issue as well.

                     

                    Actually it is also possible that for another customer a completely different rule blocks. Therefore I suggest having an SR for each customer affected and review the individual policy and find out what the problem is. We will certainly have to modify the default rules to prevent this from happening once we spotted the issue (or fix an incorrect detection for corrupted objects).

                     

                    Let me know once you got a response. I would be interested to see what is going on as well.

                     

                    Best,

                    Andre

                    • 8. Re: Google.com/.co.uk being blocked - any ideas?
                      asabban

                      Hello all,

                       

                      can anyone provide me with a feedback or configuration backup who encounters the problem with "corrupted objects" after an update? Simply upload it to ftp.webwasher.com/incoming and let me know the file name via PM.

                       

                      I would really like to replicate this and find out what is happening.

                       

                      Thank you,

                      Andre

                      1 2 Previous Next