F12 Developer Tools for Diagnosing Behind-The-Scenes Blocking and Errors

[johnaldridge]

Problems that occur behind-the-scenes can be a challenge to diagnose.  Usually, it's enough to do in the Web Gateway, but not always.  Some users are savvy enough to run Fiddler (Free Web Debugging Proxy - Telerik)  or Wireshark.

However, I often find myself directing average users to dig into F12 Developer Tools and capturing request in the F12 “Network” tab.

Microsoft has a nice introduction to this here: Using Windows Internet Explorer Developer Tools Network Capture  (Internet Explorer)

I thought I'd add a few pointers specific to diagnosing Web Gateway issues (and other web proxies), since I so often find myself walking people through this.  And, if you are a user who is hitting intermittent issues, you may want to quickly capture the errors on your own, so that you'll have something to pass on to your support staff that they can actually work with.

I'm only going to cover Internet Explorer here, but much of this also applies to Firefox and Chrome.  And, there's no reason this thread can't include posts from other proxy admins for those if needed.

---

The F12 Developer Tools of Internet Explorer includes a “Network” tool that is very useful to finding those behind-the-scenes URL's that are being blocked or running into hidden errors.  In such cases, the only way to get to the causes of those problems may be through this tool; and if the problems are intermittent, you may find yourself having to run this tool as an end user.  However technical this may seem, this brief guide should simplify the process.

Setting Up:

1. Hit that F12 keyboard button to start it, though it's also under the gear menu.
2. Click the “Network” tab on the black tab bar.
3. Pop it out into its own window using the double-window icon, , second from the right in the black bar.  F12 starts as a bottom pane by default, and that's just too cramped for long lists of requests.
4. Under older versions, you have to hit the green play button, , to start capturing, but it starts on its own in the latest versions (and naturally, the red square, , will stop it).
5. IMPORTANT: You can only capture for the browser tab from which F12 was opened; if you close that window or tab, you have to repeat the above.
6. Refresh the browser tab to capture requests associated with that tab.

The Captured Requests

1. Each line is one request.
2. The "Result/Description" column tells you if a request was successful, with 200's being good, 300's being normal redirects, and 400's and 500's are problems.
3. The "Description" may be standard text, such as “Bad Gateway” (502) or “Service Unavailable” (503), which are server problems not Web Gateway problems (yes, the Web Gateway is not always the problem)—otherwise, it might be custom text from the Web Gateway, which can be very useful to your support team.
4. Right clicking on a request allows you to copy the URL.  Please, always provide your support people with the affected URL's, and don't send only screen image images from which we'll have to type out a long URL .  Sometimes, though rarely, it will also be necessary to include the URL you requested or clicked, as redirects can complicate things.
5. If your support people request it, they may also want the “Request Headers” and/or “Response Headers”.  In older versions, you will have to click tabs for this, but it's in a pane to the right in newer versions.
6. Other details are available and may be of interest, such as “Method” and “Content type” which the Web Gateway may have acted upon.
7. While not easy to read in it's raw form, “Body” of a block page (in its footer) may include very useful details for speedy fixes.
8. There is timing information for performance problems.

Key Tips

1. From the tool bar of F12, you can clear the cache, , and clear cookies, , for the specific browser tab from F12—so you don't need to restart the browser much of the time.
2. Toggle the “Always refresh from server” button, , on to prevent use of the cache.
3. Be careful with the “Clear entries on navigate”, , as it can erase meaningful results when on, though it keeps the list from growing too large.
4. When the above is “Clear entries on navigate” is on, you can manually clear the entries with “Clear session”, .

Remember: The right amount of detail gets the fastest fix.  Both the footer and the body of a block page contain juicy details, it's never wrong to include those.  That way your support people will have the actual URL that was blocked, as well as reason it was blocked (yes, there are many reasons a thing can be blocked, and knowing exactly which one saves heaps of time).  And again, the URL you requested or clicked on may not be the one that was blocked, so provide that as well, particularly if it's different from that of the block page.

So, these are the things I end up discussing when I walk a user through a deep dive diagnostic session.  Now that I've written it out, I wondering why I haven't done this before.  Hopefully, others will find it useful to have these points laid out in advance.

If other forum members want to add tips for Firefox and Chrome, please do.

[johnaldridge]

P.S.:  The network trace results can be exported from Internet Explorer using the icon that resembles a floppy disk (or ctrl-s), and it can be exported from Chrome and Firefox by right clicking the trace results and selecting "Save all as HAR" or "Save all as HAR with content".

These can then be sent to and reviewed by support staff (note that contents are in the JSON format and can be worked with in a text editor or with scripting languages, such as Perl).