MCP - Common Logging Data Missing

[RR]

We are pulling info from our MCP database into Splunk. I can't find the following data in the database. Please point me in the right direction to find this info typical of web gateway logs.

Src IP in IPv4 format (currently the database only shows it in IPv6)

Src Port

Dest IP

Dest Port

Block reason 

HTTP User Agent

HTTP Referrer

HTTP Status

[jscholte]

Hi RR,

Wheres the data coming from? Did you mean Content Security Reporter? If you mean CSR you could just have MWG send the data directly to Splunk (and also send it to CSR). See: https://community.mcafee.com/t5/Documents/Web-Gateway-Understanding-and-Configuring-Syslog-for-your-...

What you described doesnt sound like something MCP (McAfee Client Proxy) would provide. This would have to come from Web Gateway or Web Gateway Cloud Service.

Web Gateway can log all of the fields you mentioned:

-Client.IP

-Connection.Port (source port for the client's connection to the MWG)

-URL.Destination.IP

-Proxy.Outbound.Port (source port for MWG's connection to the server) or URL.Port (destination port)

-Block.Reason

-Header.Request.Get("User-Agent")

-Header.Request.Get("Referer")

-Response.StatusCode

Web Gateway Cloud Service logs the client IP, block reason, user-agent, referer, and status code.

Best Regards,

Jon

[RR]

Yes, I meant that we are pulling the data from CSR. We already have plenty of MWGs but we'd like to see the same info from our MCP clients when they are roaming outside our web gateways.

[ams]

I am also having this problem and do not see where I can go to submit an enhancement request to get this information added. The logs coming from our users on the cloud proxies are virtually useless without the missing data.

[RR]

I agree. The logs a essentially useless without this info. I'd like to submit an enhancement request as well.

[jscholte]

Ideas can be subimitted here:

https://community.mcafee.com/t5/Business-Ideas/idb-p/business-ideas/label-name/web%20protection

I agree these things would be useful. The only fields mentioned that are not currently supported are in WGCS are:

• Referer
• Destination IP
• Connection.Port (might not be as useful for traffic exiting the Web Gateway Cloud Service)
• Destination Port can be gleened from the requested URL as it will indicate the port if not default

[jmilford]

The lack of destination IP is really impactful, can this be corrected?

[ams]

Here's my ticket. I don't know if it helps to jump on one or make multiple.

https://community.mcafee.com/t5/Business-Ideas/McAfee-Cloud-Web-Protection-Logging-Enhancement/idi-p...

[jscholte]

Making one is best then people can vote on it.

Just FYI, the latest version of the WGCS Logging API (v4) includes some of the fields you mention. The next version of CSR will parse these fields.

• Block reason
• HTTP User Agent
• HTTP Status Code

If you use the Cloud Log puller, you can adjust the API version to 4 and you'll get these new fields:

https://community.mcafee.com/t5/Documents/Web-Gateway-Cloud-Service-Cloud-Log-Puller-for-Windows/ta-...

Other fields you mentioned like Dest Port and Request Line could be assumed or compiled from other fields included in the log.

aka Request Line = {{uri_scheme}}://{{requested_host}}{{requested_path}}

and Dest Port = (uri_scheme == 'http'), then dest port = 80, same for https.

The only fields that are not included which require a development change are Dest IP and HTTP Referrer.

Fields in v4:

"user_id","username","source_ip","http_action","server_to_client_bytes","client_to_server_bytes","requested_host","requested_path","result","virus","request_timestamp_epoch","request_timestamp","uri_scheme","category","media_type","application_type","reputation","last_rule","http_status_code","client_ip","location","block_reason","user_agent_product","user_agent_version","user_agent_comment"

[ams]

Thank you! I wrote a Python version of Log Puller so I'll update that now.