cancel
Showing results for 
Search instead for 
Did you mean: 

Web Gateway: Understanding and Configuring Syslog for your SIEM

Web Gateway: Understanding and Configuring Syslog for your SIEM

Introduction

This document will outline the most common topics as it pertains to using syslog to forward access log data from McAfee Web Gateway 7.x to a syslog server. After reading this document you should have a good understanding of a) the configuration, and b) common issues encountered while using syslog for logging purposes.

Prerequisites

Before beginning to configure syslog, you need to make up your mind about what suits your environment best. Below are some items that will make the process easier.

Decide how to send

How do you want to send the data to the syslog server? Do you want to send it over UDP or TCP? Some syslog servers may not have TCP listener ports. The most common UDP listener port is 514, whereas TCP can vary from application to application. For UDP use a single @, for TCP using a double @.

UDP (most common)

# Send MWG Access events using UDP

daemon.info     @x.x.x.x:port

# Send MWG Access and Audit events using UDP

daemon.info;auth.=info @x.x.x.x:port

# Send all events using UDP

*.*     @x.x.x.x:port

TCP

# Send MWG Access events using TCP

daemon.info     @@x.x.x.x:port

# Send MWG Access and Audit events using TCP

daemon.info;auth.=info @@x.x.x.x:port

# Send all events using TCP

*.*     @@x.x.x.x:port

Decide what severity to send

The recommended severity is 6 (Info). However, for reference see below list of alternate severities:

    • 0 - Emergency (emerg) - System is unusable.
    • 1 - Alert (alert) - Action must be taken immediately.
    • 2 - Critical (critical) - Critical conditions.
    • 3 - Error (error) - Error conditions.
    • 4 - Warning (warning) - Warning conditions.
    • 5 - Notice (notice) - Normal but significant condition.
    • 6 - Informational (info) - Informational messages.
    • 7 - Debug (debug) - Debug-level messages.

For more info see: http://tools.ietf.org/html/rfc5424#page-11

Decide what to send

What kind of access log data do you want to send to the syslog server? Do you want to send all of it? Only blocked requests?

Decide what format to send

What format does your syslog server require the data to be presented in? You may want to check with your SIEM admin to see what format you should configure the MWG to send.

Default format

Accepted by McAfee Content Security Reporter v2.0 (CSR, also accepts other modified formats as well). Content Security Reporter simply requires that the format (log header) be input into it's configuration, such that it can process the syslog data accordingly.

McAfee SIEM (Nitro)

Accepted by McAfee SIEM (formerly Nitro). The McAfee SIEM format requires additional log fields that are not written in the default format.

CEF format

Accepted by SIEM's such as ArcSight. The devices parsing the output from the CEF format can be directional. In the case that a virus is found different rules are used to formulate the syslog data sent to the syslog server.

Configuring the syslog daemon (rsyslog)

Before configuring the McAfee Web Gateway rules, we will need to update the syslog daemon configuration. This should only be done in the GUI using the File Editor. This will need to be done on a per appliance basis.

Use the GUI file editor

When configuring the syslog configuration file, you may have the urge to jump on the command line and update the /etc/rsyslog.conf directly. This will not end well, it will get overwritten and you will lose all changes. You must also make these changes on a per appliance basis.

Don't write to disk

It is important that you do not write the access log data to disk (via syslog). By default, it is possible to fill the /var partition.

Look for a line similar to the following:

*.info;mail.none;authpriv.none;cron.none                /var/log/messages

Replace it with the line below:

*.info;daemon.!=info;mail.none;authpriv.none;cron.none                -/var/log/messages

This updated line will make it so the syslog daemon to not write any messages coming from the "daemon" facility (aka McAfee Web Gateway) with "info" level, to the /var/log/messages file.

Send to syslog

To send the data to a syslog server using UDP add a line similar to the line below to the end of the file (where your syslog server IP address is substituted for x.x.x.x). daemon.info represents events created by the logging rules, auth.=info represents events created by the audit logs.

daemon.info;auth.=info @x.x.x.x:514

Configuring the rules

Now that the syslog daemon has been configured, we can configure the rules in the McAfee Web Gateway to start passing messages to it. To do this, we simply need to create the contents of the message, and then configure an event to send the newly created message. As mentioned above, there are various different formats which your syslog server may require the message to be in. Below are the most common.

Default format

For the default format, all that is required, is to create a rule which applies the conditions in which you want data to be sent to the syslog server. In our case, we're sending ALL access log data to the syslog server. The only change required is to create an additional rule to send the logline to syslog.

Name: Send to syslog

Criteria: Always

Action: Continue

Event: Syslog (6, User-Defined.logLine)

McAfee SIEM (Nitro)

The McAfee SIEM (Nitro) format includes additional log fields that the McAfee SIEM (Nitro) will parse. See the Online Ruleset Library (in the Content & Cloud Security Portal) for the most up-to-date McAfee SIEM ruleset. Below is an example log entry:

McAfeeWG|time_stamp=[01/Jan/2015:02:12:31 +0800]|auth_user=jsmith|src_ip=10.10.69.1|server_ip=172.224.247.54|host=www.mcafee.com|url_port=80|status_code=301|bytes_from_client=279|bytes_to_client=1149|categories=Business, Software/Hardware|rep_level=Minimal Risk|method=GET|url=http://www.mcafee.com/|media_type=text/html|application_name=|user_agent=Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)|block_res=0|block_reason=|virus_name=|hash=|filename=|filesize=753|

Below is screenshots of the rulesets installed in the log handler:

CEF Format

The CEF format is much different from the default McAfee Web Gateway format. See the Online Ruleset Library (in the Content & Cloud Security Portal) for the most up-to-date CEF format. The CEF format will include the column metadata (i.e. what the column represents) in the log line. The CEF format is a generic format that a large number of SIEM vendors support including Arcsight and Splunk.

CEF:0|McAfee|Web Gateway|7.3.2|301|Proxy--|2|rt=Sep 02 2013 16:55:57 cat=Access Log dst=12.234.121.129 dhost=www.mcafee.com suser=jsmith src=10.10.69.1 requestMethod=GET request=http://www.mcafee.com/ app=HTTP cs3=HTTP/1.1 cs3Label=Protocol/Version cs4=Business, Software/Hardware cs4Label=URL Categories cs6=Minimal Risk cs6Label=Reputation fileType=text/html out=1182 requestClientApplication=Mozilla/5.0 Firefox/23.0 cs1= cs1Label=Virus Name cn1=0 cn1Label=Block Reason cs5=Default cs5Label=Policy

Below are some screenshots of what the rules will look like:

Attachments

Below are links to the rulesets referenced in the screenshots above. They can be imported using the Ruleset library.

McAfee SIEM (Nitro) logging ruleset

CEF syslog format ruleset

Configure Audit log to Syslog

Audit logging is used to track changes made to the Web Gateway's configuration, it also track's login's and logout's. Starting in 7.6.2, audit log entries can be send to syslog (as such, a SIEM). To enable this feature check the box for "Write audit log to syslog" under Configuration > Appliances > Log File Manager > Settings for the Audit Log.

Format

The syslog entry for the audit log is generated in the CEF format. See below example:

Nov 23 19:22:33 localhost CEF: 0|McAfee|WebGateway|1|USER_LOGIN|USER_LOGIN|3|Timestamp=23/Nov/2016:19:22:33.289 User=admin Action=USER_LOGIN Source_Type=USER Source_ID=10.10.69.2 Appliance=gsd-mwg1 User-Agent=Java/1.8.0_111 Role=Super Administrator

Facility and Severity

Audit events are sent using the "auth" facility at the informational severity (6). So rsyslog config would use auth.=info if you wanted to send this to syslog.

Common Issues

Filling MWG disk

It is common for customer's to fill their /var partition if they do not prevent MWG from writing to the /var/log/messages (see: "Don't write to disk"). To verify you are not writing to disk, run the following command:

tail -f /var/log/messages

Messages not received on syslog server

If for some reason messages are not received by the syslog server, the issue could be occurring due to Firewall restrictions on the network. To verify that the McAfee Web Gateway is sending the messages you can perform a simple tcpdump to see the packets in real-time. Other issues may stem from the rsyslog configuration file, please review Configuring the syslog daemon.

tcpdump port 514

Message size

McAfee Web Gateway will truncate the syslog message before sending it to the syslog server. This can happen if the message to be sent, is over 2000 characters. To adjust the message size the following line can be added to the rsyslog.conf:

$MaxMessageSize <size_nbr>

*Discussed in Community thread: https://community.mcafee.com/message/298694

Additional Uses

Use of syslog is not limited to forwarding access log data. Syslog can also be used for monitoring the McAfee Web Gateway's health status, using another Best Practice on Notifications and Alerting Options.

Links

Content & Cloud Online Ruleset Library - https://contentsecurity.mcafee.com/ruleset_library/

The Syslog Protocol (RFC5424) - http://tools.ietf.org/html/rfc5424

Additional rsyslog configuration parameters: http://www.rsyslog.com/doc/rsyslog_conf_global.html

Changelog

2017-12-12 - Formatted and updated branding

2016-11-23 - Added steps to send audit log to Syslog.

2014-12-31 - Updated McAfee SIEM & CEF Format sections to reference the McAfee Content & Cloud Security Online Ruleset Library.

Comments
cryptochrome

If I disable writing to /var/log/messages, will I still be able to write to the default log path? In other words: I want syslog + local logging.

Thanks

MWG will still write to the access log (/opt/mwg/log/user-defined-logs/access.log/access.log). The goal of what I wrote above was to not write to /var/log/messages, which is on the /var partition and is much smaller.

Take note of the respective events:

Syslog(6,User-Defined.logLine) - What sends the log line message to syslog daemon (used in syslog rules)


FileSystemLogging.WriteLogEntry(User-Defined.logLine)<Access Log Configuration) - What sends the log line message to a log file (used in normal logging rules)

Best,

Jon

cryptochrome

Is it possible to use other facilities than 'daemon'? Let's say I want to send out different logs (access.log, access_denied.log etc.) and have them end up in different logs on the syslog server. I usually do this by using different facilities.

haaris

Hi Everyone,

How to exclude the Kernel/system/hardware syslogs like system reboot,hard disk failure etc..

I have configured syslog at level 6.

Hi Haaris,

If you followed the guide, this should *exclude* such events. Using "daemon.info @syslogserver:514" should limit the messages sent to your syslog server.

If you wanted to send all events you would use something like "*.* @syslogserver:514".

Best Regards,

Jon

haaris

I have configured daemon.info. @syslogserver:514 in rsyslog.conf but still I m getting reboot messages/hardware/kernel messages.

cowboy71

Its a long shot but hoping someone could help!

I've configured as per the above to log from the MWG to the McAfee SIEM.

Working pretty good except for the fact that the "BytesFromClient" and "BytesTo Client" both record as "0" for all entries.

Anyone came across this before?

slizka

Doesn't work. Lots of non-existing objects.

Which part doesnt work? Or what ruleset are you importing? Please start up a new discussion thread if it might be lengthy.

slizka

Lots of stuff... can't remember... solved it by using some other ruleset from community... Hate MWG... I've never seen such undocumented crap, wonder how they can sell it as corporate solution...

ITWebSec

you're just too stupid to use it.

slizka

Yep, I'm stupid...

matthewj

We're adding a second syslog server.  Would configuring it be the same as the first, just adding  a second line of "daemon.info @syslogserver:514"?

yd9038

Yes, that's right.

"daemon.info @syslogserver:514", to send over UDP

"daemon.info @@syslogserver:514", to send over TCP so that no messages are lost

teofilov

Dear all, i´ve just configured MWG to send events to a SIEM, configuring "Write audit log to syslog" under Configuration > Appliances > Log File Manager > Settings for the Audit Log. and adding "*.* @syslogserver:514" in rsyslog.conf, but im not seeing that any audit event is going from MWG to the SIEM, (using tcpdump -i eth0 port 514) however the audit.log is getting the events OK, and, when i do tcpdump -i lo port 514, i seeing the audit events reaching localhost! please help me! im missing something else?

Thanks in advance!

Hi Matthew and YD,

If you're using two syslog servers with the same log format, then using the same config entry is fine:

daemon.info @syslogserver1:514

daemon.info @syslogserver2:514

BUT if you're sending two different log formats, you might want respective the format sent to its respective syslog server. In that case it gets a little more complex, however, its doable.

We would do something like this instead

# Format 1 (nitro), Logging rule uses severity level 6 aka Info in the Event

daemon.info;daemon.!=notice @syslogserver1:514

# Format 2 (CEF), Logging rule uses severity level 5 aka notice in the Event

daemon.info @syslogserver2:514

The "daemon.info" part means that we send everything from the "daemon" facility, with severity info or lower (0-6).

By using "daemon.info;daemon.!=notice" we're saying send everything from the daemon facility, with severity info or lower, but exclude daemon.notice specifically (the part that makes this specific is the !=).

Hope this helps!

Best Regards,

Jon

Hi Teo!

What version are you on? I did just notice there was a bug regarding the audit logging over syslog, this is fixed in 7.6.2.6:

McAfee Corporate KB - Web Gateway 7.6.2.6 Release Notes PD26796

"Audit log information could not be recorded using syslog due to a problem with parsing time zone values. (1165003)"

Best Regards,

Jon

mustapha.arakji

Hi,

I need the rule set for CEF format but this link will ask for a username and password that i can't access, anyone can provide this info to us:

CEF syslog format ruleset

vvadym

Please clarify. I have no special configuration for access.log (use default one). But when I set in /etc/rsyslog.conf daemon.info @x.x.x.x:514 I see messages in access.log format and CEF. Can't understand why access log goes to rsyslogd.

Hi vvadym,

I'd suggest creating a new thread and include some screenshots.

The only way you'd see access.log entries in syslog, is if your rules tell it to send to syslog using the event Syslog(X, User-Defined.XXX). Perhaps you have it in your rules twice? Or perhaps there is a gap in logic.

The logic gap could happen if you have a criteria tied to your CEF rule, but no logic applied to your "Send to Syslog" rule.

For example, in the CEF rules I describe above, there is no logic gap. There is two rules, one with the criteria: VirusFound equals true, another with criteria: VirusFound equals false. This accounts for both TRUE and FALSE situations. If I were to disable the one of them, then what you describe could happen.

The simple fix is to merge the send to syslog rule with the CEF rules by adding the Syslog even at the end of your events in each rule.

Hope that helps.

cburgman

Anyway to configure syslog to send audit.log with a hostname other than localhost?

This is an improvement we will release with 7.7.2 version. This is already published as Beta release in contentsecurity.mcafee.com portal.

-Sergej

slizka

This is actually what I've received from the QRadar support which I put to single line:

# send access log to qradar *.info;

daemon.!=info;

mail.none;authpriv.none;

cron.none -/var/log/messages *.info;mail.none;

authpriv.none;

cron.none

@@[IP]:514

But to be able to send access logs via syslog I guess it should be like that...?

#prevent storing on disk?

*.info;daemon.!=info;mail.none;authpriv.none;cron.none -/var/log/messages

#send accesslogs matched by above ruleset through syslog?

daemon.info @@10.110.96.56:514

Version history
Revision #:
1 of 1
Last update:
‎08-13-2013 12:31 PM
Updated by: