McAfee Web Gateway Best Practices and Common Scenarios

[#1]

Welcome!

Dear MWG Fan Community,

Now that MWG 7 has been around for a little bit and we have plenty of experience with the dos and don'ts of this most powerful web gateway ever, we figured it was time to get some best practices out there and spread the word about some of the awesome features MWG has to offer.

Below is a collection of documents written to help you understand the MWG better and hopefully cover some of the common cases you as an Admin experience.

Part of the idea is to collect feedback (No, not the feedback.zip this time ) from you as community contributors and keep improving and adding to the collection. If you have a topic that you would like to see covered or learn more about, please let us know in the comment section below.

We hope you find this collection of best practices and common cases interesting and ultimately helpful in making your admin life easier. Let us know what you think and keep comments and suggestions coming!

Your MWG Team

Getting Started

Installing your first Web Gateway

Deploying to Amazon Web Services (AWS)

Upgrade Best Practices and Understanding Release Branches​

Release Notes Listing

Configuring Automatic Backups

Understanding Central Management (Clustering)

Troubleshooting with Rule Engine Tracing

Webinars

Complete Web Protection on Your Firewall? Think Again! Top 10

SSL Scanner capabilities webinar (1hr)

Deployment Modes

Direct Proxy vs Transparent Proxy Comparison

Understanding WCCP

Understanding ProxyHA (mfend - MWG < 8.2)

Example Proxy HA configuration using HAProxy (MWG >= 8.2)

Example Transparent Proxy HA configuration using HAProxy (MWG >= 8.2)

Load Balancer Best Practices

Understanding Transparent Bridge

Understanding Reverse Proxy

Hosting the proxy.pac/wpad.dat

Troubleshooting Next Hop Proxy Issues

Web Hybrid

Configuring Web Hybrid Policy Sync

Configuring McAfee Client Proxy (MCP) for Web Hybrid​

Web Gateway Cloud Service

Introduction - What does it mean for me?​

FAQ for Web Hybrid Customers​

Configuring Site-to-Cloud Traffic Redirection (IPSec)​

Configuring SAML Authentication​

Deploying and Managing McAfee Client Proxy (MCP) with ePO Cloud​

Authentication

Choosing the right Authentication Method for your Deployment

Understanding NTLM and Windows Domain Membership

Understanding LDAP

Kerberos

Configuring Kerberos (simplified guide)

Understanding and Configuring Kerberos (extended guide)

McAfee Three Headed Dog (A Kerberos Setup Tool)

Proxy Related

Via and X-Forwarded-For Headers (Proxy Loop Prevention)

Understanding FTP over HTTP

Understanding Progress Indication Methods

Understanding and Configuring Bandwidth Control

Filtering Policy

Understanding and Optimizing your Rules

Policy Assignment - Performing filtering based on groups/user/IPs

Customizing your Block Pages

Understanding the Error Handler

Integrations

Integrating with Advanced Threat Defense (ATD)​

Configuring reporting for Advanced Threat Defense (ATD) in Content Security Reporter (CSR)​

Integrating with the Threat Intelligence Exchange (TIE) and Data Exchange Layer (DXL)​

Writing your own Playbook

Understanding URL related Properties

Understanding User-Agents

mwginternal.com - Get Creative with your Rules

Subscribed lists and how they can help with problematic connections

Subscribed Lists and External Lists Format Examples

HTTPS Inspection

SSL Scanner capabilities webinar (1hr)

SSL Scanner Rule Examples

Understanding "Client Context"

Deploying a trusted CA to your Clients

Considerations when Whitelisting HTTPS URLs

HTTPS in transparent deployments and how SNI can help

Installing and Configuring an HSM in your MWG​

Configuring SSL Tap with Network Data Loss Prevention (NDLP) Monitor​

Common Issues

Understanding HTTP 502's

Streaming Media and how the Streaming detector helps

Flash Videos (via RTMP) do not play

Logging and Monitoring

Understanding Customized Logging and Log File Management

Configuring Incident Notifications and Alerts

Configuring File System Usage Monitoring

Understanding and Configuring Syslog for your SIEM

Configuring log file Encryption and log field Anonymization

Configuring and Customizing Email Notifications

Reporting

Content Security Reporter

Configuring log file pushing to Content Security Reporter (CSR)

Web Reporter

Configuring log file pushing to Web Reporter

Understanding Page Views

Adding a custom Log Field to your Reports

Understanding Directories and Duplicate Users

Database Maintenance and Cleanup

Hardware and Appliance Maintenance

Configuring your Remote Access Card (RMM)

Collecting Hardware Logs (getlogs)

Partition Resizing

Restoring a backup after a Hardware replacement

Offline Updates for Environments with no Internet Access

Adding a Hard Drive back into a RAID array

Recommended memory upgrade for 7.5.x

Tools

Web Gateway Policy Viewer

Web Gateway PreConfig

Web Gateway Appliance Setup

Web Gateway Three Headed Dog (A Kerberos Setup Tool)​

Web Gateway Cloud Service - Cloud Log Puller for Windows (Powershell Script)​

Contact McAfee

Technical Support

Uploading Files to Technical Support

URL Feedback

URL Categorization Submissions to TrustedSource

AV Feedback

False Detection Submissions (KB62662)

Changelog

2017-12-15 - Added Load Balancer Best Practice, Cloud Log Puller

2017-12-06 - Added DXL, Memory Upgrade, direct links to Cloud Service articles, links to tools, reorganized some of the sections

2017-02-28 - Added SSL Tap and NDLP Integration link

2017-02-06 - Added Cloud Threat Detection Integration link

2017-01-27 - Added Web Gateway Cloud Serivce Expert Center link

2016-08-23 - Modified 7.5.x Memory upgrade to new link

2016-07-25 - Added Bandwidth Control guide

2015-11-12 - Added 7.5.x Memory upgrade to Hardware section

2015-01-16 - Added "Troubleshooting Next Hop Proxy Issues"

2014-12-30 - Added "Simplified Kerberos Setup", "How to gather hardware logs (getlogs)", "Policy sync with Web Hybrid", "Integration with ATD", "Setting up MWG with CSR"

2013-10-04 - Added "Introduction to Reverse Proxy", "LDAP Authentication on the McAfee Web Gateway", "Subscribed Lists and External Lists Format Examples", "Rule Engine Tracing"

2013-09-30 - Added "Sending Access logs via syslog", "mwginternal.com explained", "Automatic Backups", "Restoring your config after a hardware replacement"

2013-09-27 - Added " Offline Updates", "Customizing Block Pages", "SSL Scanner Rule Examples"

2013-09-27 - Added "Progress Indication Methods Explained", "Transparent Bridge Gotchas", "How to Roll Out a CA to your Clients", "Partition Resizing"

2013-06-27 - Added "NTLM Domain Membership", "Configuring MWG and WR", "Custom Log Field Reporting", "Group Reporting pitfall", "WR DB maintenance"

2013-06-27 - Added "WCCP Explained", "Direct vs. Transparent Proxy", "Hosting Proxy.pac", "Rule Optimization", "MCP"

2013-06-25 - Added "Error Handling", "Upgrading", "SNI explained", "FTP over HTTP"

2013-05-16 - Added "Flash videos (via RTMP) do not play"

2013-05-03 - Fixed link for "502" explained"

2013-03-29 - Added "Notifications and Alerting", "Submitting URLs" and "How Logging works"

2013-03-28 - Initial Release

[TomBryant]

Impressive amount of information collected here!

[bragot]

This was a life saver for me.  Thanks for creating it!

[olahmann]

Great collection of helpful documents. Thanks a lot!

[Troja]

Dear Support Team,

this is a great ressource.

• HA Cluster:  We tested HA Cluster with 5xMWG5500 and 18000 Users. We plugged off the network cable from the HA-Master. 🙂
  This was a requirement from customer how fast the VIP switches in case of the HA Cluster crashes or is not available.
  Result: We just los 1 Ping!!!

Debugging the Ruleset: Today the ruleengine tracing is fine, but the result files are not so easy to analyse. This takes some time. Resolving this, we always implement a Debug LOG File on MWG to figure out what is going on.

https://community.mcafee.com/thread/54270

Cheer, Thorsten

[blazej]

Keep up the good work

This is very helpful. I would love to see here other deployment scenarios maybe Proxy + WCCP?

Thanks

[Troja]

@Blazej: Today WCCP is supported only for HTTP protocol. FTP, MMS and RTSP can not be managed with WCCP.

WCCP redirect methods:

MWG to WCCP router: L2 rewrite is used

WCCP router to MWG: IP-GRE is used

You can not set the configuration using L2 rewrite for both directions.

This are the options MWG and wccp can be used.

Cheers,

Thorsten

[jebeling]

Almost..

Actually WCCP router (or switch) to MWG can be GRE or L2

Return traffic goes direct to client via available route with MWG spoofing the source IP of the website. This is neither L2 rewrite nor GRE.

[randomeclipse]

Very, very useful. Thanks for pulling it together.

[abenjami]

I know that we already have a Best Practices article here;

https://community.mcafee.com/docs/DOC-4771

(SSL Scanner Maintained Lists Bypasses)

But I figured it might be good to elaborate a little more on some of the more common bypasses I have seen in use along with how to configure them.

(This is not as "official" as the "Best Practices" but it does help cover some new list content added due to Office365, Lync etc...)

Here is the Microsoft KB from TechNet which prompted the addition of the new lists;

Office 365 URLs and IP address ranges

[mbagheryan]

abenjami wrote:

I know that we already have a Best Practices article here;
https://community.mcafee.com/docs/DOC-4771
(SSL Scanner Maintained Lists Bypasses)

But I figured it might be good to elaborate a little more on some of the more common bypasses I have seen in use along with how to configure them.
Making Bypasses for SSL Scanner using Maintained Lists
(This is not as "official" as the "Best Practices" but it does help cover some new list content added due to Office365, Lync etc...)

Here is the Microsoft KB from TechNet which prompted the addition of the new lists;
Office 365 URLs and IP address ranges

Thanks for the Provided links.

[abenjami]

Due to the positive responses I got from my last discussion post, I have put another discussion together in regards to bypassing client Antimalware updates from the Web Gateway Antimalware engine.

As before (This is not as "official" as the "Best Practices" but it does help cover some new list content added due to F-Secure, Symantec, Trendmicro etc... update servers)

[Troja]

Hi all,

how about the Data Exchange Layer / Threat Intelligence Exchange integration?? 🙂

Cheers

[jdepriest]

We just started looking at the requirements to get the Data Exchange Layer / Threat Intelligence Exchange integration going on our network. Any additional documentation would be greatly appreciated.

[Troja]

Hi ,

DXL integration is already available with MWG. 🙂

Cheers

[wan.ikbal]

brilliant!! Keep it up guys!

[jdepriest]

I was looking for a new version of Erik Elsasser's policy viewer and discovered the link is now protected.

https://community.mcafee.com/docs/DOC-2110

Access to this place or content is restricted. If you think this is a mistake, please contact your administrator or the person who directed you here.

Does anyone know what's up with that? I have version 1.4.0 and it is having trouble opening my most recent backups. I want to give someone in another group the ability to browse an archived configuration so they don't need access to a live system.

I'll probably just open a ticket with McAfee, but I wanted to see if anyone else had a similar experience.

Thanks!

JRD

[johnaldridge]

Eek, me too, and I can see that I'm logged into the forum.

[maheshs]

Great amount of core information .

[jameskhow]

Hi,

Did you manage to get a solution or able to get a copy of the policy viewer? Please share if you have it.

Thanks.

regards

James