Best Practices: Resolving Miscategorized or Uncategorized URLs

Version 1

     

    Introduction

     

    From time to time you may encounter a URL or IP address that you believe is miscategorized. The following steps will help you determine the current categorization of the URL, confirm your URL Filter Database is current, and how to quickly resolve any miscategorized URLs you find.

     

    Initial check of the URLs categorization

     

    The first step in resolving any categorization issue it to check the URL on http://trustedsource.org

    Navigate through Feedback > Check Single URL and select the McAfee Web Gateway v7.x/6.9.x (resident) product in the drop down > Check URL.

     

    There are three possible outcomes of this check:

     

      1. TrustedSource shows the same category as you saw in your environment (you still disagree with this category)
        Example: Local Environment: Pornography
                       TrustedSource:      Pornography
                                                                           >>>> Go to the "Miscategorizations" Section below


      2. TrustedSource shows the URL as uncategorized
        Example: Local Environment: Pornography
                       TrustedSource:      Uncategorized
                                                                            >>>> Go to the "Uncategorized" Section below


      3. TrustedSource shows a different category than you saw in your environment
        Example: Local Environment: Pornography
                       TrustedSource:      Business
                                                                            >>>> Go to the "Different Categorization" Section below

     

     

    Miscategorizations

     

    In the scenario where the trustedsource URL category result shows the same category as what you are seeing in your block message:

     

    1.   This type of miscategorization cannot be resolved without submitting the URL to http://trustedsource.org

        • Follow steps as described in the Trusted Source account creation and Submission process.

     

    This applies when the block message you are seeing for (example) www.google.com shows a category of Pornography and the URL category result from http://trustedsource.org also shows a category of Pornagraphy.

     

     

     

    Uncategorized

     

    If the URL is uncategorized the reason you are receiving an incorrect category in your environment is most likely coming from the IP address of the URL.

    You can check the IP using steps described below or simply submit the URL for categorization, see ‘Submitting URL’s to Trusted Source’ section below.   

     

    1.   To check IP addresses of an uncategorized URL from your MWG:

        • In the web gateway UI, navigate to Troubleshooting > Network Tools > enter host > click nslookup

     

     

    nslookup.png

     

     


        • Check the IP address returned by nslookup by browsing to http://trustedsource.org. (Feedback > Check Single URL > select the McAfee Web Gateway v7.x/6.9.x (resident) product in the drop down > Check URL)
        • You can use the following formats: (Examples: http://x.x.x.x or http://x.x.x.x/filename.ext)
        • For more information on how the McAfee Web Gateway performs the lookup for the URL/IP, see the “Additional information” Section below.

     

     

    Different Categorization

     

      • If the URL is categorized on http://trustedsource.org, but is in a different category than what you saw in your environment, it is very likely that updates have been made to the URL database already.

    Check your current URL database version in the Web Gateway Dashboard against what is shown on the results page of http://trustedsource.org.

     

     

    TrustedSource.org:

    Fig1.jpg

     

     

     

    MWG-UI:

    Fig2.jpg

     

     

     

     

      • If the URL Filter versions are different please trigger a manual update.To trigger the manual update, navigate to Configuration > Appliances > Manual Engine Update.


     

    Fig3.jpg

     

     

     

      • If after forcing the update you are still experiencing problems, please contact McAfee Web Gateway Technical Support using one of the following methods.

    Web based support portal: https://mysupport.mcafee.com

    Phone: http://www.mcafee.com/us/about/contact-us.aspx#ht=tab-techsupport

     

     

     

    Trusted Source account creation and Submission process

     

    The steps below will describe how to create an account for logging in to http://trustedsource.org 

    Creating an account for submitting Uncategorized or Miscategorized URLs:

    • Open a browser and navigate to http://trustedsource.org 
    • On the main page, under the Login section select -> Create Account
    • Complete the Registration form, select if you wish to Subscribe to live malware threat alerts and click Create Account.
    • Once the registration is processed you will receive an Account Validation email with a link to activate your account. Login with your credentials and your account is activated.

    Submitting URL’s to Trusted Source:

    • Navigate to http://trustedsource.org and login or create an account so that you can login.
    • Once your account is created, login and navigate to the Feedback drop down at the top of the page to check single URL.
    • Select your product from the drop down > select the resident option > enter the URL and click Check URL.
    • Once the categorization information has been displayed, select the categorization drop down(s) to what you would like to have the URL categorized as and click Submit.
    • After the URL is submitted you will receive a confirmation screen stating Ticket ID '#123546' you are encouraged to check all check boxes so that you can monitor the process of your URL submission and click submit again.
    • The average turnaround is typically 1 business day. Depending on the current amount on ticket submissions this time period may increase.

     

     

     

    Additional information:

     

    Below are a couple of settings that can commonly influence categorization. You can find these in the MWG UI, here:

     

    Policy > Settings > Engines > URLFilter

     

      • Do a forward DNS lookup to rate URLs:
        • This setting will check the IP of an uncategorized URL and return category/rating information based on what it finds, in the local and cloud database.

     

      • Use online GTI web reputation and categorization services if local rating yields no result:
        • If no local categorization for the URL (hostname) is found, MWG will check the GTI cloud URL Filter database for category and rating information.

     

        • If no GTI cloud categorization exists for the URL AND the “Do a forward DNS lookup to rate URLS” engine setting is enabled, the MWG will try and categorize the IP address of the requested domain locally.

     

        • If no local categorization for the IP address is found, then MWG will query the GTI cloud URL Filter database for category and rating information.

     

    If you have an issue with a URL categorization that you need to create a short term workaround to access, please review the following Community Article: https://community.mcafee.com/docs/DOC-4514

     

     

    Conclusion

     

    With this data you should be prepared to resolve most issues with Miscategorized or Uncategorized URLs.