0 Replies Latest reply on Nov 6, 2017 8:36 PM by jebeling

    Load Aware Burst to WGCS Saves Resources and Money for Hybrid Environments

    jebeling

      If you are experienced with enabling anti-malware scanning on MWG whether signature based, or the emulation based gateway anti-malware, you know that the anti-malware scanning significantly reduces the overall req/sec capacity of the appliances (physical or virtual). This makes sense because to do thorough anti-malware screening takes a lot of cycles. An appliance with SSL scanning on that is not scanning for anti-malware can typically handle 3x or more of the number of requests that it can handle when SSL scanning is on and all content that can be scanned is being scanned. While best practice is to have SSL scanning on for all but a few selected trusted categories or sites and all downloads that can be scanned, are scanned with Gateway Anti-Malware, this can result in assigning many more dedicated resources to your web gateways just to handle redundancy and short term peaks. These resources will be under-utilized at all other times.

       

      The level of resources (appliances and VMs) required to handle peaks can be dramatically reduced by monitoring the CPU load and either offloading or bursting anti-malware to the Web Gateway Cloud Service (requires WPS, SWE, or CSS licenses for all users) as described below. Or by simply blocking or bypassing anti-malware for select sites as described Load Aware Web Gateway Anti-malware Scanning and Blocking Saves Resources and Money.

       

      If you have a license for Web Gateway Cloud Service for all your users you can dramatically reduce the resources you need to purchase and maintain. The trick is to intelligently use the load awareness feature of the MWG. When CPU loading is high, you can temporarily reduce the load by using the cloud as a next hop proxy for all cycles. CPULoad is a statistics counter value that can be used in a rule criteria. Your next hop proxy selections should be based on the location of the MWGs. You will need to configure IP authentication for the public IPs that will be seen as the source for the traffic from the MWGs. An example ruleset is attached (recommended placement is after URL Filtering but before any DLP or Anti-malware rulesets) and looks like this (note you do not enable this ruleset in the cloud):

       

      Ruleset updated 11/6/2017 to include passing authentication to the cloud. Note that the Load Aware Burst to Ruleset should be placed after the SSL Scanning ruleset if SSL Scanning is enabled. Use of the authentication passing rules requires a companion ruleset that is enabled in the cloud and placed after the SSL scanning rules. As the authentication is passed in headers, authentication based rules will only work in the cloud for HTTPS sites that are scanned. Also important to note that you must add the MWG CA to your list of accepted CAs in certificate verification because the on-premise web gateway will need to accept the cert that is rewritten by WGCS.

       

      Rule Sets
      Load Aware Burst to Cloud

      [This Ruleset checks for a high load condition and if it exists the request and all
      subsequent cycles for the request will be handled by next hop to WGCS. Using this ruleset requires a subscription license to WGCS and WGCS must be configured for IP
      authentication for the public IP that will be the source of the traffic from the web gateway. Next Hop proxy settings should be set based on MWG location.]

      [✔] Enabled [✘] Disabled in Cloud
      Applies to: [✔] Requests [✔] Responses [✔] Embedded Objects
      1: User-Defined.HighLoadRequest equals true
      2: OR Statistics.Counter.GetCurrent("CPULoad")<Default> greater than or equals User-Defined.CPULoadThreshold
      Handle High Load Request
      [✔] Enabled [✘] Disabled in Cloud
      Applies to: [✔] Requests [✘] Responses [✘] Embedded Objects
      Always
      EnabledRuleActionEventsComments
      [✔] Enabled Capture Load Value and High Load as UDPs for Entire Transaction
      1: Cycle.Name equals "Request"
      ContinueSet User-Defined.HighLoadRequest = true
      Set User-Defined.CPULoad = Statistics.Counter.GetCurrent("CPULoad")<Default>
      This would also be a good place to add events to syslog or send an email to notify an
      administrator of the high load condition.
      [✔] Enabled High Load Next Hop Proxy
      1: Cycle.Name equals "Request"
      2: AND URL.Categories<Default> none in list Categories Always Processed Local°
      ContinueSet User-Defined.CPU-Load-Logline =
           "Load Bypass Antimalware (CPU Load: " +
           Number.ToString(User-Defined.CPULoad) +
           ") " +
           URL.ReputationString<Default> +
           " " +
           List.OfCategory.ToString(URL.Categories<Default>) +
           " " +
           URL +
           " " +
           Authentication.UserName +
           " " +
           IP.ToString(Client.IP)
      FileSystemLogging.WriteLogEntry(User-Defined.CPU-Load-Logline)<CPU Load Bypass>
      Next Hop Proxy<WGCS Next Hop>
      Set User-Defined.BurstToCloud = true
      Use cloud for filtering of select content when load is high.
      [✔] Enabled Pass Username to Cloud Via Header
      1: User-Defined.BurstToCloud equals true
      ContinueHeader.Add("X-Authenticated-User",Authentication.UserName)Passes Authentication information to cloud in
      headers. Must add cloud synched rules to
      remove the headers after capture.
      [✔] Enabled Pass UserGroups to Cloud Via Header
      1: User-Defined.BurstToCloud equals true
      ContinueHeader.Add("X-Authenticated-UserGroups",List.OfString.ToString(Authentication.Us erGroups,", "))Passes Authentication information to cloud in
      headers. Must add cloud synched rules to
      remove the headers after capture.
      [✔] Enabled All Other Filtering in Cloud via Next Hop Proxy WGCS
      1: User-Defined.BurstToCloud equals true
      Stop Cycle
      Handle High Load Response
      [✔] Enabled [✘] Disabled in Cloud
      Applies to: [✘] Requests [✔] Responses [✘] Embedded Objects
      Always
      EnabledRuleActionEventsComments
      [✔] Enabled Handle Response
      1: User-Defined.BurstToCloud equals true
      2: AND Response.StatusCode equals 403
      Block<Cloud Block>Set User-Defined.InTheCloud = trueThis rule to show block page from MWG indicating block was generated in cloud (next hop proxy).
      [✔] Enabled All Other Filtering in Cloud via Next Hop Proxy WGCS
      1: User-Defined.BurstToCloud equals true
      Stop Cycle


      And here is the ruleset that is used in the cloud to utilize the headers added by MWG

       

      Rule Sets
      Burst to Cloud Get Authentication From Headers
      [✔] Enabled [✔] Enabled in Cloud
      Applies to: [✔] Requests [✘] Responses [✘] Embedded Objects
      1: InTheCloud equals true
      EnabledRuleActionEventsComments
      [✔] EnabledSet User-Defined.InTheCloud
      Always
      ContinueSet User-Defined.InTheCloud = trueVariable used in block page template
      [✔] EnabledGet Authentication UserName if in header
      1: Header.Exists("X-Authenticated-User") equals true
      ContinueSet Authentication.UserName = Header.Get("X-Authenticated-User")
      Set Authentication.IsAuthenticated = true
      Header.RemoveAll("X-Authenticated-User")
      [✔] EnabledGet Authenticated UserGroups
      1: Header.Exists("X-Authenticated-UserGroups") equals true
      ContinueSet Authentication.UserGroups = String.ToStringList(Header.Get("X-Authenticated-UserGroups"),", ","")
      Header.RemoveAll("X-Authenticated-UserGroups")