cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Web Gateway: Load Balancer Best Practices

Introduction

Happy customers use load balancers. They allow for easy maintenance of your proxies and help reduce any proxy related issues from impacting your users. This guide can serve is a reference point for how Web Gateway can help with your load balancer strategy.

T1hank you for your input from the original Community discussion:

Load Balancer Modes

There are various modes that load balancers support. Below is a synopsis of the common modes and what to expect from them.

  • Direct Routing / Server Return mode = Load balancer simply routes the packets directly to the upstream Web Gateway. Client IP is obtained from the source IP in the packets.
    • Web Gateway Mode: Direct Proxy w/ Layer 2, Transparent Router
  • NAT mode = Upstream Web Gateway must be in segmented network with load balancer as the default gateway. Client IP is obtained from source IP in packets.
    • Web Gateway Mode: Direct Proxy w/ Layer 2, Transparent Router
  • Source NAT mode = Load balancer acts as a proxy and sends traffic to upstream Web Gateway must parse X-Forwarded-For to get client IP. In some configurations the X-Forwarded-For can be unreliable.
    • Web Gateway Mode: Direct Proxy

 

Web Gateway Modes

As it relates to a load balancer deployment, Web Gateway has three modes it might be deployed in when working with a load balancer:

  • Direct Proxy
    • Typically used in source NAT mode because Web Gateway is just an upstream proxy.
  • Direct Proxy w/ Layer 2
    • Typically used in high bandwidth deployments with more intelligent load balancers.
    • Accepts packets where destination IP is not it's own -- No iptables modification required!
  • Transparent Router
    • Typically used in deployments with less intelligent load balancers, bandwidth/throughput is a trade-off because Web Gateway must perform some redundancy functions which require use of the McAfee Network Driver.
      • McAfee Network Driver (mfend) has a maximum throughput of 1Gbps
    • Like Layer 2, it accepts packets where destination IP is not it's own -- No iptables modification required!

 

Load Balancer Health Checks

To check the health of the Web Gateway, a variety of health checks can be performed. These health checks could be as simple as a ping, or the load balancer could be checking the response body for certain text.

 

Ping

Pinging the Web Gateway simply validates that it's reachable, it doesn't tell us if traffic is flowing. As a result ping should be an auxiliary test performed, but not be a health check which carries too much weight.

 

TCP Port Check

Checking to see if the Web Gateway's proxy port is available is another test to validate the services are functioning on a very basic level.

 

HTTP Request

The ultimate test of whether or not the proxy is alive, is whether it can serve requests! For this purpose we've come up with some example health checks that can be used to check the status of the Web Gateway.

 

Internal Check

First example is an internal check, meaning we want to see if the Web Gateway is available WITHOUT connecting to the internet. When paired with the ruleset attached to this guide, the example request request for healthcheck.mwginternal.com will be served only from the Web Gateway -- no DNS or connection to the internet is required. This could be useful in situations where there might be a problem with the outbound internet connection, but the Web Gateway is up and running just fine. The example requests demonstrate a GET and HEAD request (HEAD requests means no response body is returned).

# URL: http://healthcheck.mwginternal.com/

 

# Load Balancer Request (\r\n = carriage return):

GET / HTTP/1.0\r\nHost: healthcheck.mwginternal.com\r\nUser-Agent: Proxy Health Check Internal\r\nConnection: Close\r\n\r\n

 

# Example cURL Command:

curl -vi --proxy http://YOUR-MWG:9090 -H "Proxy-Connection: Close" -H "User-Agent: Proxy Health Check Internal" http://healthcheck.mwginternal.com/

 

# Request from Load Balancer to Web Gateway:

GET / HTTP/1.0

Host: healthcheck.mwginternal.com

User-Agent: Proxy Health Check Internal

Connection: Close

 

# Response Web Gateway to Load Balancer:

HTTP/1.0 200 OK

Connection: Close

 

McAfee Web Gateway

 

# URL: http://healthcheck.mwginternal.com/

 

# Load Balancer Request (\r\n = carriage return):

HEAD / HTTP/1.0\r\nHost: healthcheck.mwginternal.com\r\nUser-Agent: Proxy Health Check Internal\r\nConnection: Close\r\n\r\n

 

# Example cURL Command:

curl -vi --proxy http://YOUR-MWG:9090 -H "Proxy-Connection: Close" -H "User-Agent: Proxy Health Check Internal" -X HEAD http://healthcheck.mwginternal.com/

 

# Request from Load Balancer to Web Gateway:

HEAD / HTTP/1.0

Host: healthcheck.mwginternal.com

User-Agent: Proxy Health Check Internal

Connection: Close

 

# Response from Web Gateway to Load Balancer:

HTTP/1.0 200 OK

Connection: Close

 

External Checks

The second kind of check would be to see if the Web Gateway's outbound internet connection is alive. If it isn't maybe it's time to pull the Web Gateway from the pool, or alert in order to take corrective action. The below examples are requests to commonly used URLs, most are captive portal URLs used by Windows, Apple, or McAfee products to validate if the machine has access to the internet. Each of them return 200 response codes simple strings in the body that can be examined by the load balancer in order to determine a status.

 

Microsoft's Network Connectivity Status Indicator: https://technet.microsoft.com/en-us/library/cc766017(v=ws.10).aspx

# URL: http://www.msftncsi.com/ncsi.txt

 

# Load Balancer Request (\r\n = carriage return):

GET /ncsi.txt HTTP/1.0\r\nHost: www.msftncsi.com\r\nUser-Agent: Proxy Health Check External\r\nConnection: Close\r\n\r\n

 

# Example cURL Command:

curl -vi --proxy http://YOUR-MWG:9090 -H "Proxy-Connection: Close" -H "User-Agent: Proxy Health Check External" http://www.msftncsi.com/ncsi.txt

 

# Request from Load Balancer to Web Gateway:

GET /ncsi.txt HTTP/1.0

Host: www.msftncsi.com

User-Agent: Proxy Health Check External

Connection: Close

 

# Response from Web Gateway to Load Balancer:

HTTP/1.0 200 OK

Content-Type: text/plain

Content-Length: 14

Connection: Close

 

Microsoft NCSI

 

McAfee's Captive Portal Check URL:

# URL: http://mcp.webwasher.com/test/MCP.txt

 

# Load Balancer Request (\r\n = carriage return):

GET /test/MCP.txt HTTP/1.0\r\nHost: mcp.webwasher.com\r\nUser-Agent: Proxy Health Check External\r\nConnection: Close\r\n\r\n

 

# Example cURL Command:

curl -vi --proxy http://YOUR-MWG:9090 -H "Proxy-Connection: Close" -H "User-Agent: Proxy Health Check External" http://mcp.webwasher.com/test/MCP.txt

 

# Request from Load Balancer to Web Gateway:

GET /test/MCP.txt HTTP/1.0

Host: mcp.webwasher.com

User-Agent: Proxy Health Check External

Connection: Close

 

# Response from Web Gateway to Load Balancer:

HTTP/1.0 200 OK

Content-Type: text/plain

Content-Length: 7

Connection: Close

 

McAfee

 

Apple's Captive Portal Check URL:

# URL: http://captive.apple.com/hotspot-detect.html

 

# Load Balancer Request (\r\n = carriage return)

GET /hotspot-detect.html HTTP/1.0\r\nHost: captive.apple.com\r\nUser-Agent: Proxy Health Check External\r\nConnection: Close\r\n\r\n

 

# Example cURL Command:

curl -vi --proxy http://YOUR-MWG:9090 -H "Proxy-Connection: Close" -H "User-Agent: Proxy Health Check External" http://captive.apple.com/hotspot-detect.html

 

# Request from Load Balancer to Web Gateway

GET /hotspot-detect.html HTTP/1.0

Host: captive.apple.com

Connection: Close

User-Agent: Proxy Health Check External

 

# Response from Web Gateway to Load Balancer

HTTP/1.0 200 OK

Content-Type: text/plain

Cache-Control: max-age=30, must-revalidate

Content-Length: 69

Connection: Close

 

<HTML><HEAD><TITLE>Success</TITLE></HEAD><BODY>Success</BODY></HTML>

 

ICAP Health Check

If you're using Web Gateway as an ICAP server it will also be useful to check the health of the ICAP service. No rules are required for this to function.

 

OPTIONS Request

To check basic availability we'll send an OPTIONS request to the Web Gateway, this will just tell us if the service is available, and what capabilities it supports.

# Load Balancer Request (\r\n = carriage return)

OPTIONS icap://YOUR-MWG ICAP/1.0\r\nUser-Agent: Proxy Health Check ICAP\r\n\r\n

 

# ICAP Request from Load Balancer to Web Gateway

OPTIONS icap://YOUR-MWG ICAP/1.0

User-Agent: Proxy Health Check ICAP

 

# ICAP Response from Web Gateway to Load Balancer

ICAP/1.0 200 OK

Methods: REQMOD, RESPMOD

Options-TTL: 3600

Encapsulated: null-body=0

Max-Connections: 100

Preview: 30

Service: McAfee Web Gateway 7.7.2 build 24546

ISTag: "00005977-14.38.44-00008745"

Allow: 204

 

Health Check Rule Set

To ensure that the load balancer health checks receive priority when passing through the Web Gateway, we've created a Proxy Health Check Rule Set to handle the HTTP Request health checks.

 

Import the Rule Set and Template

 

1. Create an "(Empty)" Template Collection and Import the "Proxy Health Check - Service Unavailable Template" zip file into the (Empty) collection.

 

2. Import the Proxy Health Check Rule Set and move it to the top of your rules.

 

3. Review the "Proxy Health Check URLs" list and add or remove any URLs as needed. The default URLs (as described in the health checks examples) are commonly used by other devices and are suitable for our needs here.

 

Maintenance Mode

The second rule in the ruleset is used for putting Web Gateway (single or multiple) into maintenance mode, meaning it will be pulled from the load balancer pool as long as it's in that list (and the load balancer is properly configured). Simply add the Web Gateway's hostname into the list, and it will send a 503 HTTP response code along with a different HTTP response body. Configuring maintenance mode allows for easy administration, as you won't need to pester the load balancer administrators everytime you need to update the Web Gateway's.

 

When a request is made to a Web Gateway in Maintenance Mode, the response would return a HTTP 503, with "Service Unavailable" as the response body:

# Request from Load Balancer to Web Gateway:

GET / HTTP/1.0

Host: healthcheck.mwginternal.com

User-Agent: Proxy Health Check Internal

Connection: Close

 

# Response from Web Gateway to Load Balancer:

HTTP/1.0 503 ServiceUnavailable

Connection: Close

Content-Type: text/html

Cache-Control: no-cache

Content-Length: 19

 

Service Unavailable

 

Conclusion

We hope this helps make your life easier and better your understanding of how Web Gateway can work with your load balancer.

Labels (1)
Attachments
Comments

thanks for another great document.

Contributors
Version history
Revision #:
2 of 2
Last update:
‎03-20-2018 11:10 AM
Updated by:
 

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community