I have an issue with CISCO ESA (former Ironport) logging style and it seems some other people had it before. This solutoin sends around 8 useful log lines through Syslog to the Receiver (it send more but the rest can be ignored). All the events share only one single field: Message_ID.
I created a correlation rule with the Group by Message_ID field and I get a correlated event for each email sent but the Correlated event custom fields are populated with the fields from the last event from those 8 (the one regarding the attachment) so I don't see the sender, recipient or subject of the email. If I expand the Source events I can see all of them but I'd like to see the custom fields populated with the content of each event ("sender" from the first event, "recipient" from the second event and so on...)
Is it possible or is just a product limitation? I've seen this done in another SIEM.
Nevertheless to say that in case of multiple recipients, Ironport send the event containing the "To" field in multiple lines, one per each recipient which make the work even harder.
As a last option, I was thinking to use an intermediary Syslog server, to write the events in a flat file and use the SIEM Collector to combine the multi line events into one single event and send it to Receiver.
Any feedback would be appreciated.
I've seen this issue before where a data source is sending multi-line syslog events. I'm not personally aware of a way that the SIEM can parse multiple syslog messages into a single event, which is what I think you're looking for instead of using group by in a correlation rule for this specific use case.
I've used NXlog in between the data source and the SIEM receiver to parse the multi-line syslog messages and output them as a single message. The documentation for the multi-line NXlog configuration is here: https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_multiline
Note that you may have to use the im_tcp input module and send events via TCP instead of UDP, unless you're only going to be sending this one ESA's logs via UDP to NXlog, as NXlog treats all UDP messages using the im_udp input module as a single source.
The reason that you're not seeing the events is probably they are being aggregated together. It's possible to use custom aggregation or to disable aggregation on a per parsing rule basis, but all SIEM's do aggregate. You can select the event that you want to de-aggregate in the UI, select Show Rule and then set the Aggregation to disabled and roll out the policy.
Average aggregation rate is ~10-15:1 so that means you basically increase the EPS for the unit by 10-15*the number of logs that are not being aggregated so you just want to be mindful about making a change that increase the EPS beyond what the unit can handle. Thanks.
I am afraid you didn't understand the issue. If you create a correlation rule to trigger when 3 events are happening in a specific order (sequence), the correlated event inherits only the values from the third event ignoring the values from the first 2 events.
The correlated event will have all the fields populated with the values from the third event only.
The only way to do it is to have an intermediary server to first collect the values from all the 3 events and combine them in one single event, then write them in a flat file and finally using the file collector to read the logs and store the values in one single SIEM event.
I case of CISCO ESA the 3 events have different fields populated.
You are correct that at this time SIEM does not support Multiline syslog. Can I suggest adding a Product Idea for supporting multiline syslog data sources?
You may want to add one about modifying correlation results to collect information from multiple events to output into the returned correlation rule.