3 Replies Latest reply on Dec 11, 2013 11:21 AM by msiemens

    Web Reporter processing - ignore log file lines

    msiemens

      I'm setting up a new WR log source and would like to filter out some of the log file noise based on multiple criteria. I've defined one criteria to filter out the "not authenticated" lines and would like to also filter out some IP addresses.

       

      In the "Processing" section, I entered the following expression into the "Ignore log file lines that match..." box where xx.yy.zz are valid IP octets:

       

           \b407\b|xx\.yy\.zz\.1[0-2]

       

      This doesn't seem to work. Is there a problem with this expression? How do I enter an "or" condition?

       

      Mike

        • 1. Re: Web Reporter processing - ignore log file lines
          sroering

          There is a round-about way to easily test your expressions without having to import logs.

           

          1) Get a few sample log lines for testing. bot postive and negative.

          2) Under Administration > Setup > Log Sources > Custom Rule Sets,  click Add

          3) Under the "Rules" put your regex in the replace coloumn. Then put $1 or $2, etc. in the "With" column.

          4) Add different regex patterns

          5) At the bottom, put a log line into the "test string" box and click "show match"

           

          It will show you which pattern matched, and what part of the line matched according to your regex.  It will also validate your regex, by throwing an error if you have a mistake.

           

          Regarding your regex, I think you are missing .* at the beginning and the end.  Keep in mind that your pattern is for the entire log line, so you need to include wildcards on both sides.

           

          I'm not sure about the or condition. I couldn't get the regex to validate using it, but Web Reporter will already ignore HTTP status code 407 messages.

          1 of 1 people found this helpful
          • 2. Re: Web Reporter processing - ignore log file lines

            If you are using MWg7.x, You also have the option of just not logging those values by creating a rule above the log writing to stop rule set.

             

            You could stop logging the 407 noise with:

             

            Response.StatusCode equals 407 OR

            Client.IP is in rangelist 1.2.3.0/24

             

            Action: Stop Rule Set

             

             

            The IPrange is just an example, you could make it as targeted as you want.

            • 3. Re: Web Reporter processing - ignore log file lines
              msiemens

              I didn't realize that it matched the entire line. Three or four years ago, I was told to simply use "\b407\b" to filter the not-authenticated log log lines so I thought that simply appending "|xxx\.yyy\.zzz\.1[0-2]" would work.

               

              In the end, I modified the log handler to stop rule set when Client.IP matches xxx.yyy.zzz.1[0-2].