cancel
Showing results for 
Search instead for 
Did you mean: 

Web Gateway: Understanding User-Agents

Web Gateway: Understanding User-Agents

Introduction

You might be asking yourself, "Okay, why do I care about the user-agent?".  Well, the user-agent can be important and can help you out in lots of situations.  Lets say iTunes keeps prompting users to enter in their credentials, or a windows media player stream simply will not play.  You might need to bypass these programs from certain rules to get them functioning the way they should be.  You can easily do that with 1 header, and that is the user-agent header.

What is a User-Agent?

A User-Agent is a header in an HTTP request that identifies the client software originating the request.

Why is this header important?

This is important because it identifies what piece of software may be making a request so you can properly identify where it is coming from.  This could be anything from the browser, to windows media player.  They all have a user-agent to identify what software it is.

Why would I need to create a rule to match on the user-agent?

There are times where you need to match the user-agent criteria for a rule.  These times usually include bypassing a specific rule set within web gateway that may be causing issues with the user-agent.  The beauty about Web Gateway 7 is that you don’t have to bypass all of the rules.  Say you are having an issue where a specific user-agent is not working with authentication (which is one of the most common times) so you only want to bypass authentication.  You can easily do that.  There will be some examples of this and other common scenarios at the end.

How do I find out what the user-agent is for a request?

There are a few ways to find out this sort of information.  Below I will list and explain some common ways to find this out:

Access Logs

It is possible to log the user-agent in your access logs.  By default, the web gateway logs the user agent to the access logs.

Example entry:

Figure C.png

Online Resources

There are lots of online resources available to find out the user-agent.  A lot of them are made publicly available.  It is possible to do a Google search, and there is also a very useful website that has a list of a lot of the common user-agents.  This URL is http://www.useragentstring.com.  Also, if you would like to find out what your current browser user-agent is you can go to this website:  http://whatsmyuseragent.com

TCP-Dump

It is possible to find the user-agent header for a GET request in a TCP Dump by following the TCP Stream of a request.  You can do this using a packet tracing software, such as ‘Wireshark’.  You would do this by finding the specific request you want, right clicking on it, and selecting “Follow TCP Stream”.  This would then bring up a window that shows the request and its headers.  You can also use a filter within Wireshark to search for a specific user-agent.  For example, you could use a filter such as:

http.user_agent matches "Mozilla"

The web gateway has capabilities to gather a TCP Dump so that you can open it with Wireshark to analyze.  For help with gathering a TCP Dump please check this KB article:

https://kc.mcafee.com/corporate/index?page=content&id=KB75056

Examples

Here are some common user-agents that we come across quite frequently:

Firefox 16.0.1

Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1  **All Firefox browser user-agents will have “Firefox/” and then the corresponding version number**

Internet Explorer 10

Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)  **All IE browser user-agents will have “MSIE” and then the corresponding version number**

Google Chrome 25

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22  **Note that there is "AppleWebKit" in the Google Chrome user-agent, not to be confused with an Apple\iPhone header**


Windows Media Player 10

Windows-Media-Player/10.0.0.xxxx

NSPlayer/10.0.0.xxxx WMFSDK/10.0

**Windows Media Player will always use both of these user-agent strings**

iTunes 9.1.1

iTunes/9.1.1

Safari on the iPhone

Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7  ** Most iPhone apps will have “iPhone” somewhere in the user-agent string**

Note:  Most browser user-agents have “Mozilla/” at the beginning, but do not confuse this with Mozilla Firefox.  This does not mean the user-agent is for Firefox, it is just what all browser user-agents start with.

How do I create a rule based on the User-Agent header in Web Gateway?

There are several reasons for why you might want to create a rule to match a user-agent.  You might want to either whitelist or block a specific user-agent.  For example, if you didn’t want users to use the Internet Explorer 6 browser, you could block the user-agent.  Another reason is that some user-agents do not know how to respond to proxy authentication.  Some common ones we have encountered in the past that have trouble with authentication are: Silverlight, iTunes, Windows Media Player, RealPlayer, Adobe Updater, Adobe Flash.  These user-agents typically do not know how to respond to a 407 proxy authentication request and simply do not answer when the web gateway asks for it, so they need to be bypassed.

The web gateway does not have a specific property for the user-agent so it needs to be entered manually.

    • First you would select the header.request.get property from the list

    • Then, you would press the “parameters” button

    • Now, here is where you have to manually type in:  User-Agent

    • Next, you have a choice to either have it match a single User-Agent, or have it match in a list of user-agents.  The choice is yours of which you want to do, but lists are always recommended because then if you have new entries you want to add later, all you have to do is add it to the already existing list.  Here is an example of a list of user-agents it can match in.

Figure D.png

    • The action you want to choose is completely up to you.  You can either block it, stop rule set, or stop cycle.  It all depends on what you are trying to accomplish with the rule.

Conclusion

Hopefully this has given you a better understanding of what the user-agent header is exactly, and what purpose it has.  Also, hopefully this gets you on the right path to creating a rule based on the user-agent property to use within your web gateway appliance.

Rule Set Examples

Bypass a user-agent from authentication

Example1.png

Bypass a user-agent from the SSL Scanner

Example2.png

Create a Global Block list of User-Agents

Example3.png

TCP Dump Troubleshooting Examples

Here is an example of what you might see in a TCP Dump that shows when a user-agent doesn't respond to proxy NTLM authentication requests, and also how to find the user-agent in a GET request.

Program is not responding to NTLM authentication

Here you can see that iTunes keeps trying to go to the iTunes store, and we keep asking for authentication.  However, each time, iTunes keeps making the same request and never responds with any credentials.  In a normal situation it would complete the NTLM handshake by first sending back the same request with an NTLM negotiate, which the web gateway would then respond back with an NTLM challenge, and then iTunes would respond back with an NTLM authenticate and provide the NTLM credentials to the web gateway.

iTunes TCP Dump.png

How to find the user-agent in a GET request

Here you can see I have selected a GET request in a TCP Dump, and under the "Hypertext Transfer Protocol" section below you can see the User-Agent header.  I did this using a program called "Wireshark".

User-Agent Header TCP Dump.png

Comments
apellepa

It will be great if you add information how to change User-Agent string...

asabban

Hello,

MWG has Header.Remove and Header.Add events. You can use those events to modify a header. Generally I would not recommend to change the header, as websites or services may be optimized for the User-Agent that is requesting content. Changing the User-Agent header could break the appearence of functionality of a web site.

Best,

Andre

edwardkayle

Would like to much obliged for imparting this with us..I also wanna know more about how two factor authentication in Google apps works?

jmhernando

I cann't see "User-Agent" in https request; is it possible get "User-Agent" in a https connection?.

Thanks,

Jose.

asabban

Hello,

if you're MWG is deployed in explicit proxy mode (e.g. proxy server is configured in the users browser) the browser will send a "CONNECT" request which contains a User-Agent header. If you are running in transparent proxy mode the browser directly starts talking SSL without a plain-text request with meta data, so you need SSL Scanner enabled in order to decrypt the SSL traffic and see the User-Agent header.

Best,

Andre

jmhernando

I'm using Client Proxy with Explicit Proxy Authentication.

When I access to https URL the User-Agent property don't appear (I attach a screenshot).

User-Agent.png

Thanks!.

Hi JMhernando,

I just saw this now, but McAfee Client Proxy is technically a transparent proxy as it intercepts traffic and forwards it to MWG. It is not an explicit proxy request like when you set the browser settings. This is why there is no user-agent.

Best Regards,

Jon

Version history
Revision #:
1 of 1
Last update:
‎03-21-2013 12:07 PM
Updated by: