Best Practices: Configuring Syslog on Web Gateway 7.x

Version 5

     

    Introduction

     

    This document will outline the most common topics as it pertains to using syslog to forward access log data from McAfee Web Gateway 7.x to a syslog server. After reading this document you should have a good understanding of a) the configuration, and b) common issues encountered while using syslog for logging purposes.

     

     

    Prerequisites

     

    Before beginning to configure syslog, you need to make up your mind about what suits your environment best. Below are some items that will make the process easier.

     

    Decide how to send

     

    How do you want to send the data to the syslog server? Do you want to send it over UDP or TCP? Some syslog servers may not have TCP listener ports. The most common UDP listener port is 514, whereas TCP can vary from application to application. For UDP use a single @, for TCP using a double @.

     

    UDP (most common)

     

    # Send MWG Access events using UDP
    daemon.info     @x.x.x.x:port
    
    # Send MWG Access and Audit events using UDP
    daemon.info;auth.=info @x.x.x.x:port 
    
    # Send all events using UDP
    *.*     @x.x.x.x:port
    
    

     

    TCP

     

    # Send MWG Access events using TCP
    daemon.info     @@x.x.x.x:port
    
    # Send MWG Access and Audit events using TCP
    daemon.info;auth.=info @@x.x.x.x:port
    
    # Send all events using TCP
    *.*     @@x.x.x.x:port
    

     

     

    Decide what severity to send

     

    The recommended severity is 6 (Info). However, for reference see below list of alternate severities:

      • 0 - Emergency (emerg) - System is unusable.
      • 1 - Alert (alert) - Action must be taken immediately.
      • 2 - Critical (critical) - Critical conditions.
      • 3 - Error (error) - Error conditions.
      • 4 - Warning (warning) - Warning conditions.
      • 5 - Notice (notice) - Normal but significant condition.
      • 6 - Informational (info) - Informational messages.
      • 7 - Debug (debug) - Debug-level messages.

     

    For more info see: http://tools.ietf.org/html/rfc5424#page-11

     

     

    Decide what to send

     

    What kind of access log data do you want to send to the syslog server? Do you want to send all of it? Only blocked requests?

     

     

    Decide what format to send

     

    What format does your syslog server require the data to be presented in? You may want to check with your SIEM admin to see what format you should configure the MWG to send.

     

    Default format

    Accepted by McAfee Content Security Reporter v2.0 (CSR, also accepts other modified formats as well). Content Security Reporter simply requires that the format (log header) be input into it's configuration, such that it can process the syslog data accordingly.

     

    McAfee SIEM (Nitro)

    Accepted by McAfee SIEM (formerly Nitro). The McAfee SIEM format requires additional log fields that are not written in the default format.

     

    CEF format

    Accepted by SIEM's such as ArcSight. The devices parsing the output from the CEF format can be directional. In the case that a virus is found different rules are used to formulate the syslog data sent to the syslog server.

     

     

    Configuring the syslog daemon (rsyslog)

     

    Before configuring the McAfee Web Gateway rules, we will need to update the syslog daemon configuration. This should only be done in the GUI using the File Editor. This will need to be done on a per appliance basis.

     

    Use the GUI file editor

     

    When configuring the syslog configuration file, you may have the urge to jump on the command line and update the /etc/rsyslog.conf directly. This will not end well, it will get overwritten and you will lose all changes. You must also make these changes on a per appliance basis.

     

    1.0.0.4a_file_editor.png

     

     

    Don't write to disk

     

    It is important that you do not write the access log data to disk (via syslog). By default, it is possible to fill the /var partition.

     

    Look for a line similar to the following:

     

    *.info;mail.none;authpriv.none;cron.none                /var/log/messages
    

     

     

    Replace it with the line below:

     

    *.info;daemon.!=info;mail.none;authpriv.none;cron.none                -/var/log/messages
    

     

     

    1.0.0.5_file_editor.png

     

    This updated line will make it so the syslog daemon to not write any messages coming from the "daemon" facility (aka McAfee Web Gateway) with "info" level, to the /var/log/messages file.

     

    Send to syslog

     

    To send the data to a syslog server using UDP add a line similar to the line below to the end of the file (where your syslog server IP address is substituted for x.x.x.x). daemon.info represents events created by the logging rules, auth.=info represents events created by the audit logs.

     

    daemon.info;auth.=info @x.x.x.x:514
    

     

     

    Configuring the rules

     

    Now that the syslog daemon has been configured, we can configure the rules in the McAfee Web Gateway to start passing messages to it. To do this, we simply need to create the contents of the message, and then configure an event to send the newly created message. As mentioned above, there are various different formats which your syslog server may require the message to be in. Below are the most common.

     

    Default format

    For the default format, all that is required, is to create a rule which applies the conditions in which you want data to be sent to the syslog server. In our case, we're sending ALL access log data to the syslog server. The only change required is to create an additional rule to send the logline to syslog.

     

    Name: Send to syslog

    Criteria: Always

    Action: Continue

    Event: Syslog (6, User-Defined.logLine)

     

    1.0.0.1_syslog_rule.png

     

     

    McAfee SIEM (Nitro)

     

    The McAfee SIEM (Nitro) format includes additional log fields that the McAfee SIEM (Nitro) will parse. See the Online Ruleset Library (in the Content & Cloud Security Portal) for the most up-to-date McAfee SIEM ruleset. Below is an example log entry:

     

    McAfeeWG|time_stamp=[01/Jan/2015:02:12:31 +0800]|auth_user=jsmith|src_ip=10.10.69.1|server_ip=172.224.247.54|host=www.mcafee.com|url_port=80|status_code=301|bytes_from_client=279|bytes_to_client=1149|categories=Business, Software/Hardware|rep_level=Minimal Risk|method=GET|url=http://www.mcafee.com/|media_type=text/html|application_name=|user_agent=Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)|block_res=0|block_reason=|virus_name=|hash=|filename=|filesize=753|
    

     

     

    Below is screenshots of the rulesets installed in the log handler:

     

    1b.png 1a.png

     

     

    CEF Format

     

    The CEF format is much different from the default McAfee Web Gateway format. See the Online Ruleset Library (in the Content & Cloud Security Portal) for the most up-to-date CEF format. The CEF format will include the column metadata (i.e. what the column represents) in the log line. The CEF format is a generic format that a large number of SIEM vendors support including Arcsight and Splunk.

     

    CEF:0|McAfee|Web Gateway|7.3.2|301|Proxy--|2|rt=Sep 02 2013 16:55:57 cat=Access Log dst=12.234.121.129 dhost=www.mcafee.com suser=jsmith src=10.10.69.1 requestMethod=GET request=http://www.mcafee.com/ app=HTTP cs3=HTTP/1.1 cs3Label=Protocol/Version cs4=Business, Software/Hardware cs4Label=URL Categories cs6=Minimal Risk cs6Label=Reputation fileType=text/html out=1182 requestClientApplication=Mozilla/5.0 Firefox/23.0 cs1= cs1Label=Virus Name cn1=0 cn1Label=Block Reason cs5=Default cs5Label=Policy
    

     

    Below are some screenshots of what the rules will look like:

     

    1.0.0.3.0a_syslog_cef_rule.png 1.0.0.3.2a_syslog_cef_rule.png

     

     

    Attachments

     

    Below are links to the rulesets referenced in the screenshots above. They can be imported using the Ruleset library.

     

    McAfee SIEM (Nitro) logging ruleset

     

    CEF syslog format ruleset

     

     

    Configure Audit log to Syslog

    Audit logging is used to track changes made to the Web Gateway's configuration, it also track's login's and logout's. Starting in 7.6.2, audit log entries can be send to syslog (as such, a SIEM). To enable this feature check the box for "Write audit log to syslog" under Configuration > Appliances > Log File Manager > Settings for the Audit Log.

     

     

    Format

    The syslog entry for the audit log is generated in the CEF format. See below example:

    Nov 23 19:22:33 localhost CEF: 0|McAfee|WebGateway|1|USER_LOGIN|USER_LOGIN|3|Timestamp=23/Nov/2016:19:22:33.289 User=admin Action=USER_LOGIN Source_Type=USER Source_ID=10.10.69.2 Appliance=gsd-mwg1 User-Agent=Java/1.8.0_111 Role=Super Administrator
    

     

    Facility and Severity

    Audit events are sent using the "auth" facility at the informational severity (6). So rsyslog config would use auth.=info if you wanted to send this to syslog.

     

    Common Issues

     

    Filling MWG disk

     

    It is common for customer's to fill their /var partition if they do not prevent MWG from writing to the /var/log/messages (see: "Don't write to disk"). To verify you are not writing to disk, run the following command:

     

    tail -f /var/log/messages
    

     

     

    Messages not received on syslog server

     

    If for some reason messages are not received by the syslog server, the issue could be occurring due to Firewall restrictions on the network. To verify that the McAfee Web Gateway is sending the messages you can perform a simple tcpdump to see the packets in real-time. Other issues may stem from the rsyslog configuration file, please review Configuring the syslog daemon.

     

     

    tcpdump port 514
    

     

    1.0.0.6_syslog_tcpdump.png

     

     

    Message size

     

    McAfee Web Gateway will truncate the syslog message before sending it to the syslog server. This can happen if the message to be sent, is over 2000 characters. To adjust the message size the following line can be added to the rsyslog.conf:

     

    $MaxMessageSize <size_nbr>
    

     

    *Discussed in Community thread: https://community.mcafee.com/message/298694

     

     

    Additional Uses

     

    Use of syslog is not limited to forwarding access log data. Syslog can also be used for monitoring the McAfee Web Gateway's health status, using another Best Practice on Notifications and Alerting Options.

     

     

    Links

     

    Content & Cloud Online Ruleset Library - https://contentsecurity.mcafee.com/ruleset_library/

     

    The Syslog Protocol (RFC5424) - http://tools.ietf.org/html/rfc5424

     

    Additional rsyslog configuration parameters: http://www.rsyslog.com/doc/rsyslog_conf_global.html

     

     

    Changelog

     

    2016-11-23 - Added steps to send audit log to Syslog.

    2014-12-31 - Updated McAfee SIEM & CEF Format sections to reference the McAfee Content & Cloud Security Online Ruleset Library.