3 Replies Latest reply on Oct 20, 2011 6:48 AM by dstraube

    Uncategorized in Trustesourced and Web Gateway

    maitane

      Good morning, We are detecting anomalous behavior with our web gateway, where users access pages that are not categorized in the Trustesourced. Here´s an example: The domain www.cienciaspuras.com is uncategorized in Trustesourced, well. When the application enters the URL Filtering RuleSet we have, we´re doing match with the rule:  Block URLs in Whose Category Blocklist is unauthenticated. And we received the following response to match. Correo electronico del usuario: URL: http://www.cienciaspuras.com/ Categorias: Pornography Usuario: Unknown (10.168.10.12) Razon de bloqueo: URL filtered Fecha de notificacion: 2011-10-20 11:34:54 Nombre de la regla: Block URLs Whose Category is in Category BlockList Unauthenticated Nombre del appliance: HZKWSG-EJ00 How is it posible that if the domain has not categorize the behavior of the MWG are doing me in that rule match and also tell me that pornography is a category? Thanks & Regards Maitane

        • 1. Re: Uncategorized in Trustesourced and Web Gateway
          dstraube

          Hello maitane,

           

          you are correct, the URL cienciaspuras.com is not listed in the Trustesource Database. So you normally would expect that it won't be blocked.

           

          By default MWG has enabled the URL Filter option "Do a forward DNS lookup to rate URLs", which you can find under Policy -> Settings -> Engines -> URL Filter.

           

          This means that MWG will not only query the domain name, but also the IP Address of the server, in this case 89.248.110.26.

           

          If you do a Trusted Source Lookup for http://89.248.110.26 you receive:

           

            URL Status Categorization Reputation
          http://89.248.110.26Categorized URL- PornographyMinimal Risk

           

           

          That's the reason why this URL is blocked. You can whitelist the domain for URL Filtering or disable the forward DNS lookup if you still want to allow access to that site.

           

          Regards,

           

          Dirk

           

           

           

          • 2. Re: Uncategorized in Trustesourced and Web Gateway
            maitane

            Thank you very much Dirk.  you are right, if I disable the DNS forward lookup everything works fine. The whitelisting solution does not help me as our users mostly access to uncategorized pages. I have another question, what would be the reason why a domain is not categorized as such, but the ip of the domain is?, I guess that will come from other possible services are being provided from that IP.

            • 3. Re: Uncategorized in Trustesourced and Web Gateway
              dstraube

              Hello maitane,

               

              maitane wrote:

               

              I have another question, what would be the reason why a domain is not categorized as such, but the ip of the domain is?, I guess that will come from other possible services are being provided from that IP.

               

              There could be several reasons why the IP is listed, but the domain name is not:

               

              - Other content on the webserver that was categorized by trusted source. Probably still available.

              - Shared webhosting service. One webserver can host multiple domains. So multiple websites are on the same server with the same IP. The webserver controls the content based on the incoming request. Often just requesting such a site with the IP Address only will display a template or will not show any content at all, so this is often not an issue. It depends on the webspace provider and the configuration of the web service.

              - The IP Address was assigned to a different server before, which hosted content leading to the categorization.

               

              It's hard to tell what the real reason was.

               

              Regards,

               

              Dirk