8 Replies Latest reply on Dec 6, 2010 7:41 AM by nqe

    webgateway security log lacking url information

      Hello all,

       

      I'm tasked with using some of the webgateway logs.  It seems to me that the security log (web gateway 6.8.7 build 8846) is missing some vital information to be of permanent good use. Most of the entries lack the (complete) url. What good is a log if it only logs the outcome and not the input/request??

      I hope this has something to do with the version running (web gateway 6.8.7 build 8846). Could someone please check and confirm that we just have to upgrade to a newer version to be able to properly use the security logs?

       

      19837:[29/Nov/2010:11:46:58 +0100] "Script /templates/adconion/js/scripts.js" 200 text/javascript js pass
      19838:[29/Nov/2010:11:46:58 +0100] "Script /templates/adconion/js/videoproducts.js" 200 text/javascript js pass
      19839:[29/Nov/2010:11:46:58 +0100] "Script " 200 text/javascript - pass
      19840-[29/Nov/2010:11:46:58 +0100] "ObjectID=clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" 200 - - pass
      19841-[29/Nov/2010:11:46:58 +0100] "Object /pix/flash/blog_banner_fr.swf" 200 application/x-shockwave-flash swf pass
      19842:[29/Nov/2010:11:46:58 +0100] "Script " 200 text/Javascript - pass

      19654-[29/Nov/2010:11:46:18 +0100] "ObjectID=clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" 200 - - pass
      19655-[29/Nov/2010:11:46:18 +0100] "Embed flash/vision.swf" 200 application/x-shockwave-flash swf pass

      19394-[29/Nov/2010:11:44:49 +0100] "Script " 415 text/html - filter
      19395:[29/Nov/2010:11:44:49 +0100] "Script " 200 text/javascript - pass
      19396:[29/Nov/2010:11:44:49 +0100] "Script " 200 text/javascript - pass
      19397-[29/Nov/2010:11:44:51 +0100] "Script " 200 - - unknown

      19791-[29/Nov/2010:11:46:38 +0100] "Script " 415 linkedin/control - filter

       

       

      While it is possible:

      19387:[29/Nov/2010:11:44:49 +0100] "Script http://pagead2.googlesyndication.com/pagead/show_ads.js" 200 text/javascript js pass

       

      Defense in Depth is more than trusting the web gateway has done the 'needfull'

       

      Anyone able to clarify my issue?

       

       

      Thanks a lot, nqe

        • 1. Re: webgateway security log lacking url information
          mcafee-com-user

          Hello,


          on MWG 6.8.7. build 8378 i cannot find a security log. Is this a new feature in bulld 8846?

          • 2. Re: webgateway security log lacking url information

            Sorry, can't tell. I had never seen the interface of a webgateway before ;-)

             

            But the location would be -> "Reporting > Overall reporting > Log File management > Activate Log Files"

            • 3. Re: webgateway security log lacking url information
              trishoar

              Hi Nqe,

               

              The information you are likely looking for is in the access.log

              if you wish to customise the security log to include the full URI then you can add this atribute to the logging

              req_line

              This will log the full request with every log entery.

               

              Regards,

               

              Tris

              • 4. Re: webgateway security log lacking url information

                It seems that that is actually not the case.

                 

                I can grep for a specific string from the security log (note down the exact time) and then am never able to find that specific entry in the access log including searching day before & after :-(

                 

                For instance from the security log: ($less security1012022358.merged-00.15.17.b9.25.4c.log)

                [02/Dec/2010:07:52:33 +0100] "Script /ScriptResource.axd?d=2Tcda_4hNKnG_MQmRortX7flZiOHfytU2ASd_utHF7jNnHRjv293aj4ae 5YXJQ5RdwWI8qQOXG0_xbHAzyD6DEtoPTQVigUkj-BBPxArA2wenJJ92Ikn7rzNcrj3XZ_teEutG85Ne NCcNy4nFysVzykiroYO2e61xSSKUBP6a441&t=ffffffffd2572c05" 200 text/javascript axd pass

                 

                Then searching for part of the string in the accesslogs for that day: nothing! (even searching day before & day after, don't know why though, but it is not there!)

                $grep "ScriptResource.axd?d=2Tcda_4hNKnG_MQmRortX7flZiOHfytU2ASd_utHF7jNnHRjv293aj4ae 5YXJQ5RdwWI8qQOXG0" access101202*

                 

                 

                So "No" that information is not in there...

                 

                For some entries the security log shows the url, for others it won't.... ~Weird science

                 

                 

                Message was edited by: nqe on 3/12/10 2:11:22 PM
                • 5. Re: webgateway security log lacking url information

                  Quite often the security.log reports values that are inside of a requested object.

                   

                  For example, the access log might show that you requested /archive.zip, but as the files inside the archive are scanned, you will see entries in the security.log like /archive.zip/file.exe.

                   

                  The file.exe would not be in the access.log but it would be in the security.log.

                   

                  So, a html page that contains a link to a script might be reported in security.log because the link itself is being scanned on the outer html page, but the script itself is never requested and would not be in the access.log, maybe because you never clicked on the link that that does the requesting of that URL.

                   

                  I hope that explains it a little better.

                  • 6. Re: webgateway security log lacking url information

                    e²,

                     

                    I doubt this is the case here, .axd & .js they get requested by loading the page, same for the .swf file.

                    These files are NOT in some kind of archive.

                     

                    If scipts do not get requested, there's no need to scan them, is there?

                     

                    Sorry, you did not convince me here. I hope something comes up to the surface.

                    • 7. Re: webgateway security log lacking url information

                      Let's take these security.log entries as an example:

                      #time_stamp "object_id" status_code media_type extension media_type_status
                      1: [03/Dec/2010:10:53:52 -0500] "GET
                      http://www.foxnews.com/ HTTP/1.1" 200 text/html - unknown
                      2: [03/Dec/2010:10:53:53 -0500] "Script " 200 text/javascript - pass
                      3: [03/Dec/2010:10:53:53 -0500] "Script /js/hbx_1.js" 200 text/javascript js pass

                      4: [03/Dec/2010:10:53:53 -0500] "GET http://www.foxnews.com/js/hbx_1.js HTTP/1.1" 200 application/x-javascript js unknown

                       

                      Line 1:
                      This is the request for the index page of the site. It contains various <script></script> tags in the page.

                       

                      Line 2:
                      This is a scan of the content inside of the first page it represents the scan of the content between the <script> tags on the page itself.


                      <script type="text/javascript">
                      ew_enableRefresh();
                      function ew_enableRefresh() {var secs=600;window.refreshInterval=setInterval(function(){location.reload(false);} ,secs*1000);}
                      function ew_disableRefresh() {clearInterval(window.refreshInterval);}
                      </script>

                       

                      Line 3:
                      This is only a scan of the content between the <script></script> tags. The entire line on the index page looks like this:

                       

                      <script type="text/javascript" src="/js/hbx_1.js"></script>

                       

                      This does not indicate that the script was actually downloaded. There is no GET request yet performed for the actual script. It is only the what is in the tags.

                       

                      Line 4:
                      This is where the object gets requested and downloads. This is where the scanning for it occurs.

                      HOWEVER...
                      You may not always see the scanning of the requested object as in Line 4 because:
                      a) It may have been blocked from another filter, like URL filtering.
                      b) The browser may already have the object in its local cache and the content was not retrieved because it didn't need to do so.
                      c) The web cache may already have scanned it and does not need scanned again because signatures haven't updated.
                      d) The actual URL for the object is not the same name as what you may think it is. JavaScript manipulation may have converted the URL to an entirely different request string.

                      e) There could be a white list or bypass entry somewhere else in the system. (like an ICAP bypass)

                       

                      In your case, the "Script, Object, Embed" lines of your logs only indicate that the data between the <script></script>, <object></object>, <embed></embed> tags.
                      It does not prove that the actual request to those src= attributes was actually performed. If the access log does not show they were performed...they weren't through this proxy.

                       

                       

                      on 12/3/10 10:42:28 AM CST
                      • 8. Re: webgateway security log lacking url information

                        e²,

                         

                        Thanks for you explaining answer, appreciated!

                         

                        Would you imply that then you would almost always see the request to the index page? (Which it seems i don't see these requests)

                         

                        If i understand correctly, every event with no other info than 'pass' is in-page javascript?

                         

                        Only when it says "GET" it is a script which actually makes the download request. Correct?

                         

                        "Script " 200 text/javascript - pass" is always preceded by a GET somewhere earlier on in the log?

                         

                        By just analyzing the security*.log it is quit impossible to tie the requesting client to the entry, if necessary the format of the security log needs to be adapted to include workstation address/name, this information is not something that can be gained from the access*.log?

                         

                         

                        Thanks for your help,

                         

                        nqe