9 Replies Latest reply on Jun 15, 2012 1:18 AM by asabban

    MWG 7.1.6.1.0 - 403 Forbidden in one specific Site

    pedro.tavares

      Hi,

       

      I'm troubleshooting an issue with this site:  http://www.musifex.pt/

       

      For some unknow reason, it returns the following error:

       

      Forbidden

      You don't have permission to access / on this server.

      Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

       

       

      i've disabled all rules that could deny the connection, then captured the tcp logs in MWG and it also returns the same error.

       

      From tcpdump:

       

       

      GET / HTTP/1.1

      Via: 1.1 192.168.69.35 (McAfee Web Gateway 7.1.6.1.0.12742)

      Host: www.musifex.pt

      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

      Cookie: e63a8b5664d1e9627b5c598f5aaadaa8=09303e2aacc28799a9e0e3059d0143ca; virtuemart=0103f12d005b880ab52658f8f4eed598

      Connection: Keep-Alive

      User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0

      Accept-Encoding: gzip, deflate

      Accept-Language: pt-pt,pt;q=0.8,en;q=0.5,en-us;q=0.3

      X-Forwarded-For: 192.168.69.115

       

      HTTP/1.1 403 Forbidden

      Date: Thu, 24 May 2012 15:06:48 GMT

      Server: Apache

      Content-Length: 321

      Keep-Alive: timeout=5, max=100

      Connection: Keep-Alive

      Content-Type: text/html; charset=iso-8859-1

       

      <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
      <html><head>
      <title>403 Forbidden</title>
      </head><body>
      <h1>Forbidden</h1>
      <p>You don't have permission to access /
      on this server.</p>
      <p>Additionally, a 404 Not Found
      error was encountered while trying to use an ErrorDocument to handle the request.</p>
      </body></html>

       

      Can anyone please test the access thought MWG? does anyone have an ideia on what could be the problem?

       

      Thank you.

        • 1. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
          trishoar

          Hi Pedro,

           

          The 403 and 404 are comming from the webserver and not the proxy.

           

          An example 403 on WW7 looks like this

           

          [root@server ~]# curl -x 10.134.144.99:80 playboy.com -I

          HTTP/1.1 403 URLBlocked

          Via: 1.1 10.134.144.99 (McAfee Web Gateway 7.2.0.13081)

          Content-Type: text/html

          Cache-Control: no-cache

          Proxy-Connection: Keep-Alive

           

          Note it returns a block category in the 403 message.

           

          I think there is an issue with the webserver, as though it works when accessing with out a proxy server, it does not work when trying webwasher 6, 7.1, 7.2 or bluecoat sg 5.5 and it times out when trying curl direct to the server.

           

          Tris

          1 of 1 people found this helpful
          • 2. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
            pedro.tavares

            Hi Tris,

             

            Thank you for the feedback. Having the results with Bluecoat and Webwasher versions it's good information on my troubleshooting. Althougt while working with the direct access is a hard fact to my customer.

             

            Again, thank you very much for the info.

            • 3. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
              asabban

              Hello,

               

              the Server does not like the "X-Forwarded-For" header MWG adds. Use a rule as below to solve the problem:

               

              Auswahl_036.png

               

              Best,

              Andre

               

              PS: This one should work as well: https://contentsecurity.mcafee.com/ruleset_library?q=50011

               

              Nachricht geändert durch asabban on 25.05.12 02:28:22 CDT
              • 4. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                pedro.tavares

                Hi Andre,

                 

                This did solve the problem.

                 

                How did you realize that this was the problem? Is there any information in logs that point to the correct solution or it was related to your expertise in HTTP communications? I'm asking this because I didn't find anything that would point to this solution.

                 

                Thank you very much for your help, it was very very helpfull.

                 

                Regards,

                 

                Pedro

                • 5. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                  asabban

                  Hi Pedro,

                   

                  It was an assumption :-) Basically what I did was the following:

                   

                  - Copied the URL

                  - Browsed it with a browser which has MWG as a proxy enabled -> Saw the 403

                  - Browsed it with the same browser but without MWG -> Page worked

                   

                  Because the response I saw when browsing through MWG was a message that was generated at the server (not by MWG) I knew that the server responds in a different way when there is MWG in the loop. If you add a proxy, usually the only difference for the server is maybe the source IP address (even this is not true when there is a NAT in place (which usually is)), and the request headers. The only headers MWG adds by default are Via and X-Forwarded-For. Usually on the last proxy within your company you should remove the X-Forwarded-For header, as it contains information about the proxies you have passed, and this becomes available to the web server. But many customers do not do so and I have already seen one or two web servers acting really weird when there are additional headers they do not expect.

                   

                  So I tried to remove both headers and browsed through MWG again. The page worked fine so I removed the removal of the Via Header. The page still worked. Then I removed the rule that removes the X-Forwarded-For header and refreshed and I encountered the 403 again.

                   

                  The X-Forwarded-For header itself is find and there is no problem with having it enabled. That the server sends such a response is something that goes wrong on the server side (maybe a very strict IPS system or a broken/malconfigured web server). For MWG all is perfect. The request the browser sends is fine, the request MWG sends to the server is fine and even the response is perfectly fine. MWG does not know that there *should* not be a 403, but content coming back. For MWG this means the user has requested something the web server does not want to present, but the forbidden message is basically an acceptable response.

                   

                  Therefore it is hard or almost impossible to have such a behavior trigger an alarm that could help an administrator to troubleshoot. There are many reasons for 4xx status codes on the web server, and I would say 99% are perfectly fine, so you would have a lot of false alerts :-)

                   

                  It is always good to think about "what does the proxy change" compared to an access without the proxy. I have seen probably a dozen of occurences where (on the remote end) servers start acting weird if they do not get the request they expect. It seems that many servers and web applications do not carefully test with a browser in the loop. Imagine you build a web site... you access it with a browser and if it works - fine. You do not think or care about proxies in the loop, so I can understand this.

                   

                  I have seen the following:

                  - Server delivering wrong response (your case)

                  - Server not responding at all, but keeping the connection open until tcp timeout kicks in (a while ago in the community)

                  - Application running on server not sending any data causing MWG to time out before application finishes (application should send some bytes to keep connection alive)

                  - Web application sending the same response in an endless loop (odd!)

                  - Application stripping content from the response

                   

                  This kind of "weirdness" is hard to find, but in many cases it is not a wrong behaviour on MWG and we can find a workaround.

                   

                  I hope this little "overview" helps you to understand what happens. And as usual support and community will assist in such cases :-)

                   

                  best,

                  Andre

                  1 of 1 people found this helpful
                  • 6. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                    pedro.tavares

                    Hi Andre,

                     

                    This overview was very helpfull, really. Very well explained,too.

                    I know I can count on support and community but I do a lot's of troubleshooting (this and other Mcafee products) and knowing how things are solved, is gold to me.

                     

                     

                    Best regards,

                     

                    Pedro

                    • 7. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                      asabban

                      Hi Pedro,

                       

                      you are welcome :-) I am happy to hear that.

                       

                      Best,

                      Andre

                      • 8. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                        cestrada

                        Is it possible to create a list of specific URL's or domains and add the Remove Proxy headers to only apply to the list?

                        • 9. Re: MWG 7.1.6.1.0 - 403 Forbidden in one specific Site
                          asabban

                          If you follow the rule that has been shown above simply change the criteria from "Always" to "URL" matches in list XYZ or "URL.Host" matches in list ABC and create a list with the URLs you would like to apply the rule to. Pretty simple :-)

                           

                          Best,

                          Andre