1 2 Previous Next 10 Replies Latest reply on Apr 3, 2012 3:30 PM by epop

    Webgateway with Webreporter

    epop

      Hi

       

      I have set up a webgateway 7 to log to webreporter. I can see in the log files on the gateway that is has been pushed successfully. I can also see the logs files  on the reporter however no data shows up on the quick view  on the reporter and no data are on the reports. I use an external sql  which is showing as connected , I have tried using local DB as well.

       

       

      Any Ideas?

       

      Thanks

      Ep

        • 1. Re: Webgateway with Webreporter
          sroering

          First check the jobs and their status.

           

          Administration > Setup > Log Sources > Jobs

           

          check if the status is "waiting" or not.  If there are a large number of jobs in the waiting status, then it's likely a parsing issue. In that case, if you have a directory, edit the directory and check the settings using the test button.  After that, send me a screenshot of the processing tab for the log source, and a screenshot of (Administration > Options > Performance > Database statistics)

           

          If there are no jobs, then you should check the configuration on each side.. make sure the push is using the correct URL and credentials.

           

          If the jobs are successful, check that their processed stats do not show 100% errors.  That would indicate the log header doesn't match the body, or there is a mistake in your logging rule in the gateway.

           

          If the jobs are failed, then either the file isn't an access log, or there was a problem with the header.

           

          Those should be enough to get you started. 

          • 2. Re: Webgateway with Webreporter
            epop

            Hi

            Thanks for getting back , From your reply I think it may be down to the log header doesnt match the body (But could be the mistake in the logging rule as well ) Some jobs are showing as successfull and other failed (these are not access files) I have attached the screen shot requested , But I can see its 100% errors . Do you have any ideas how to slove the issue with the log header matching the body

             

            Thanks for your help so far I really appreciarte it .

             

            Ep

            • 3. Re: Webgateway with Webreporter
              sroering

              So I can see a few problems.

               

              1) Since the failed jobs are not access logs, you are using the global push under (Configuration > Appliances > Log File Manager).  You should disable pushing from here, but leave the log rotation and deletion configuration since it still applies to non access logs.  Then set the access log push configuration under the policy.  (Policy > Settings > Engines > File System Logging > "Access log Configuration"). The last name depends on your config.

               

              copy and paste the header used for the access log configuration I just mentioned (for each access log configuration).  Hopefully you have one access log configuration for every physical appliance in your cluster.

               

              Then I also need a screenshot of the rule for the access log.  (Policy > Rule Sets > Log Handler > "Access Log").  Select rule, and click "Show details", I need the stuff in the events column. You should also have a rule for every appliance in the cluster that maps to an access log configuration mentioned above.  Then set a criteria for each rule to match on the "System.HostName".  It may seem a little strange, but it's the recommended way to do it. This guide may help you with the configuration.

               

              ftp://anonymous:anonymous@ftp.support.securecomputing.com/outgoing/MWG7+WR.doc

              • 4. Re: Webgateway with Webreporter
                epop

                Thanks for the reply and sending on the document , very helpfull . I have stopped the global push, only access log files are now showing as succesfull. Attached is the screen shots requested . I havent set up a rule  or set criteria  for each applaince but I can see the benifits and will do so once I get this working. The jobs are still showing as 100% ERRORS so hopefully you will spot somehitng

                 

                Thanks again

                 

                EP

                • 5. Re: Webgateway with Webreporter
                  sroering

                  Here you go.

                  correct_config.bmp

                   

                   

                  And regarding the separation of logs, it's necessary for performance.  You may not experience any side effects with two or three appliances, but if you are pushing logs from multiple appliances, Web Reporter caching system won't work right and you'll see log parsing performance issues. As the number of appliances increase, performace will degrade.

                   

                  Message was edited by: sroering on 4/3/12 1:29:48 PM CDT
                  • 6. Re: Webgateway with Webreporter
                    epop

                    no luck yet

                     

                    my current set up

                     

                     

                    DateTime.ToWebReporterString

                    " ""

                    Authentication.UserName

                    "" "

                    "Authentication.Attributes"

                    "" "

                    String.ReplaceIfEquals (IP.ToString (Client.IP), "", "-")

                    " "

                    String.ReplaceIfEquals (Number.ToString (Response.StatusCode), "", "-")

                    " ""

                    Request.Header.FirstLine

                    "" "

                    List.OfCategory.ToString (URL.Categories)

                    "" ""

                    String.ReplaceIfEquals (URL.ReputationString, "", "-")

                    "" ""

                    MediaType.ToString (MediaType.FromHeader)

                    "" "

                    String.ReplaceIfEquals (Number.ToString (BytesToClient), "", "-")

                    " ""

                    Header.Request.Get ("User-Agent")

                    "" ""

                    List.OfString.ToString (Antimalware.VirusNames)

                    "" ""

                    Number.ToString (Block.ID)

                    "" ""

                    • 7. Re: Webgateway with Webreporter
                      sroering

                      Sorry.  But that should show that it's easy for anybody to make mistakes. So I missed the extra line you have related to the legacy property for user groups (in red below).   I'd recommend just removing them since that would be a complete list of groups for every line for that user.  Not something most people would write in the access log.   Web Reporter won't use them for the group reporting anyway, that is a Web Reporter config.

                       

                      If you insist on keeping groups in your logs, you might want to check if you see "Authentication.UserGroups" which is the current name if you are on the latest 7.1.6 line.  Then you would need a second quote in the line between attributes and client IP.  Then you also need to add something for the header, like "user_groups" between "auth_user" and client_ip

                       

                       

                       

                      DateTime.ToWebReporterString

                      " ""

                      Authentication.UserName

                      "" "

                      "Authentication.Attributes"

                      "" "

                      String.ReplaceIfEquals (IP.ToString (Client.IP), "", "-")

                      " "

                      String.ReplaceIfEquals (Number.ToString (Response.StatusCode), "", "-")

                      " ""

                      Request.Header.FirstLine

                      "" "

                      List.OfCategory.ToString (URL.Categories)

                      "" ""

                      String.ReplaceIfEquals (URL.ReputationString, "", "-")

                      "" ""

                      MediaType.ToString (MediaType.FromHeader)

                      "" "

                      String.ReplaceIfEquals (Number.ToString (BytesToClient), "", "-")

                      " ""

                      Header.Request.Get ("User-Agent")

                      "" ""

                      List.OfString.ToString (Antimalware.VirusNames)

                      "" ""

                      Number.ToString (Block.ID)

                      "" ""

                       

                      Message was edited by: sroering on 4/3/12 2:26:41 PM CDT
                      • 8. Re: Webgateway with Webreporter
                        epop

                        I tried removing it , still the same , Sorry I can open  SR if I am taking to much of your time. Thanks

                         

                        DateTime.ToWebReporterString

                        " ""

                        Authentication.UserName

                        "" "

                        String.ReplaceIfEquals (IP.ToString (Client.IP), "", "-")

                        " "

                        String.ReplaceIfEquals (Number.ToString (Response.StatusCode), "", "-")

                        " ""

                        Request.Header.FirstLine

                        "" "

                        """

                        List.OfCategory.ToString (URL.Categories)

                        "" ""

                        String.ReplaceIfEquals (URL.ReputationString, "", "-")

                        "" ""

                        MediaType.ToString (MediaType.FromHeader)

                        "" "

                        String.ReplaceIfEquals (Number.ToString (BytesToClient), "", "-")

                        " ""

                        Header.Request.Get ("User-Agent")

                        "" ""

                        List.OfString.ToString (Antimalware.VirusNames)

                        "" ""

                        Number.ToString (Block.ID)

                        "" ""

                        • 9. Re: Webgateway with Webreporter
                          sroering

                          Sorry, last line.. Should just be one quote.  It's using [quote, space, quote].  The space (probably not a problem), but the second quote is un matched.

                           

                          I really hope that is the last problem.. 

                          1 2 Previous Next