2 Replies Latest reply: Mar 8, 2013 7:56 PM by cscoup8 RSS

    Question on MWG7 performance tips


      I am curious to know if any of the following make a difference in terms of improving performance.  I realize that many of these probably have a negligible impact but I am still curious:



      • Does the order of items in a list matter (i.e. should the most frequently encountered values go on top) or does MWG7 optimize this on its own based on frequency count or sorting algorithms.


      • In the HTML Opener engine settings, does it make a difference to reduce the amount of HTML elements (node name records) to the least amount required?  The default has many more than what I need.


      • Am I better off deleting rules that I'm likely not going to use instead of disabling them (strictly for performance reasons, not for tidying things up)


      • If I have a ruleset with run criteria "Always" that runs on a request and response cycle that has multiple rules within it, of which there is one rule that technically only needs to run on a request cycle, am I better off moving it into its own ruleset, or keeping it where it currently sits and adding the "Cycle.Name = Request" criteria to that one rule.


      • If I have a ruleset in which the last rule is there to set a property value, should the rule action be stop ruleset or continue?


      • Is there a recommended amount where one should use a list instead of multiple OR statements in the rulecriteria (i.e. are you better off using "URL.Host = A or URL.Host = B or URL.Host = C" versus a list that has A,B and C)
        • 1. Re: Question on MWG7 performance tips



          I will try to provide some hints. Please understand that I cannot give an estimate  about how "big" a performance impact may be. If the system is not highly loaded or you have an incredible amount of objects you will most likely not notice any changes in the performance. However it definitely makes sense to build everything with keeping the performance in mind to be safe in the future or when peaks occur.


          1.) The order of  items in a list does matter. If you have a very big list with lets say 5.000 entries and every request is checked against this list you will notice a big difference when you sort the list by "popularity". You should always try to estimate which list entries (or criteria in a rule set/rule) may hit very often, and place these entries close to the top.


          I had one example where MWG was deployed for multiple customers. There were one or two big customers causing 90% of the traffic and a few dozens of other smaller customers. Then there was a list of rules applies which tried to find the customer name (based on source IP) and apply some actions. With the big customers at the end of the list MWG had to perform dozens of checks for all the requests form the big customers, which didn't make any sense.


          If you can assume that some entries will be hit more than others, make sure that MWG hits them first :-)


          2.) For every HTML Element that is in the setting MWG will cause a new "embedded" cycle and run the HTML element through the rule engine, applying all rules that are valid for the embedded cycle. So yes, this will make a difference. Reduce it to the minimum of HTML elements you would like to check.


          3.) From a performance perspective I do not think this will make a big difference. I believe if you disable a rule MWG will notice this and skip it, which is a very very simple lookup and not very expensive from a performance perspective. However if you have 1000 disabled rules walking through the rule engine may be slightly slower (probably not noticable) compared to not having those rules at all. I would leave everything disabled that you would like to enable again some day.


          If you want to just keep something because maybe you want to add it again or want to use it as a template you could export the rule group to the rule library and delete it. So it is still available for import, but does not hang around useless in the production policy.


          4.) I don't think this will make a big difference. In this case I would design the rules in a way that they make more sense from a readability perspective, rather than worrying about performance. It does not make sense to optimize your policy for performance so much that you can't read/understand your rules any longer.


          5.) It does not make a difference. I would use "Continue" or "Stop Rule Set" depending on the rule and the question "is it likely that further rules will be added below, and if so, do I want to trigger (Continue) or skip (Stop Rule Set) them?".


          6.) No recommended amount. You can basically feel free what you want to do. However I find "is in list" far more readable than a couple of ORs. So I usually prefer to create lists when


          - I have more than 3 entries

          - I am uncertain whether more entries will come later


          For example if I want to whitelist everything for Facebook and the associated URLs it does not make sense to say URL = facebook.com or URL = fbcdn.akamai.net or ..., but I would create  a list of "Facebook Hosts" so you can see and read the rule very quickly and understand what it does. As long as it is just a simple decision or something that does not fit in a list (Command.Name = GET OR Command.Net = HEAD) I would prefer using the criteria rather than a list.


          The most important note is that you place "cheap" decisions to the top, and "expensive" decisions to the bottom. Especially blacklist or whitelist entries should be placed on top. URL filter decisions is also done pretty quickly, while AV scanning takes its time. So if there is any rule that could prevent you from filtering for viruses (because it is blocked anyway), place it in front of the virus filter rule sets. If you watch out for those "bigger" performance traps you should be pretty safe.




          • 2. Re: Question on MWG7 performance tips

            Excellent information.  Thank you for taking the time to write it.