Content Security Reporter: Adding a custom Log Field to your Reports

[#1]

• Introduction
• Overview
• Example using destination IP

Introduction

Content Security Reporter has default data points for reporting. If you want additional info, there are four custom columns available for reporting on additional data. However, these additional columns have some restrictions.

• Only available in detail data / not available in summary data
• No special data type recognition. For example, byte values will not calculate sum or average values, IP addresses cannot be filtered by a netmask, etc.

For general information about setting up Web Gateway and Content Security Reporter log sources, please see this article: https://community.mcafee.com/docs/DOC-4928​

Overview

There are 3 primary steps for getting Content Security Reporter to report on additional info.

1. Configure the Web Gateway to include the desired field in the access log.
2. Update the Web Gateway’s access log header to reflect the change made in step one.
3. On Content Security Reporter, modify your log source by adding a User-Defined Column for this new log header, so CSR can understand this particular access.log format.

Example using destination IP

We often see that administrators want to run reports based on, or to at least include the destination IP, so we will use destination IP as an example for adding a custom column.

**Before going any further, it is important to know that if there is any misconfiguration on the Web Gateway side, then none of your access logs will be processed by Content Security Reporter until this is corrected. Any affected logs may not be able to be repaired. Therefore, we recommend testing the changes on a practice access log first. Instructions for creating a practice log can be found here, under the “Creating a Customized log” section:

More information about customizing and managing your log files on Web Gateway, please see this article:  https://community.mcafee.com/docs/DOC-4812

Configure the Web Gateway to include the desired field in the access log

In the Web Gateway UI go to: Policy > Log handler (bottom left corner) > Access log...highlight Write Access.log rule and click edit so the edit rule window appears.

In the Edit Rule window, click on 4. Events,highlight the "Set User-Defined.logline" and click edit.

"Edit Set Property" Window appears: click the lower Add button in this window - below where it says "To concatenation of these strings"

"Enter a string" Window appears: Enter double quotes space double quotes and click OK. This space is one way to separate additional fields.

Next, repeat the same step as above however this time select Use Property and from drop down box select IP.ToString(IP). With IP.ToString(IP) highlighted, Click on "Parameters" to the right of it.

Select "Parameter Property" (top Right corner) and from the search field type URL.Destination.IP. Select the property and click OK.

*Important: before proceeding, stop right here and have a look at your event column and note where this new log line has been placed. It should be at the very end and it should read:

+space

+IP.ToString (URL.Destination.IP) -- as seen in the two screenshot below:

*To ensure that the new lines are place in the correct place. Highlight both lines by holding and pressing Ctrl and clicking the last two lines. Then use the Move up button and place it below Application.ToString (Application.Name). See screenshot below.

*Do NOT save your changes just yet, as the header needs to be modified. Continue with step II below.

Update the Web Gateway’s access log header

We must now modify the header, so it matches the order of your events column as seen above. Since our new log column is last in the events list, it must be last in the headers line, label it dest_IP.

In the Web Gateway UI go to:

Policy > Settings > File System Logging > Access Log Configuration... under "File System Logging Settings" you will see the Log header box -- add server_ip to the end of it.

As an example, I was using the default Write access.log rule, so the header for it would now look like this:

time_stamp "auth_user" src_ip status_code "req_line" "categories" "rep_level" "media_type" bytes_to_client "user_agent" "virus_name" "block_res" “dest_IP”

Following the advice below about headers will prevent much frustration, as ANY type of error with headers will prevent Content Security Reporter from understanding the log format, and it will not process any corresponding logs.

• Other than underscore (_), no other special characters are permitted
• Header names cannot contain spaces. Use an underscore wherever a space is desired (dest_IP)
• If the field logged was wrapped in double quotes, then wrap the header name in double quotes as well
• Avoid duplicate names – do not give it name that already exists in the Log header 

On Content Security Reporter, modify the log source by adding a User-Defined Column for the new log header

Now that the Web Gateway is logging the destination IP and the access log header has been updated, you must modify your log source inside of Content Security Reporter. Note that, if you have multiple Web Gateway log sources in Content Security Reporter, you have to make sure to update all of them.

Add a User-Defined Column for this new log header (dest_IP) so that CSR can use this particular access.log format and so that you can report using it. To modify the header, you can take the following steps:

• Log into your Content Security Reporter

• Navigate to Administration -> Setup ->Log Sources -> Log Sources.

• Select your log source, and hit ‘Edit’.

• On the ‘Edit Log Source’ screen, click the ‘User-Defined Columns’ tab.

• Assuming you aren’t already using it, check the checkbox for User-Defined 1 to ‘Populate this column’, and enter a Log file header of dest_IP (assuming of course, you used dest_IP as your header on the MWG side).

Now, any further log files coming into Content Security Reporter from this log source should have this new log entry parsed and added to the User-Defined 1 column. You can use this when running advanced reports on detail data. It’s important to note that all data prior to when this change was made will not have this data available to report on.