cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Parsing multi line events

I'm trying to write a custom parser for a small application that we use called Robo-FTP.  The log files are a bit difficult for me to nail down and get parsed.

A sample log entry looks like this;

Fri Oct 05 08:35:26 => *Sending file "\\Aps_image_1\CG\CSS\PROD\FTP\STATUSES\FLIGHTDECK\CUSTNAME\TRANSMIT\10050833.850.pgp"
Fri Oct 05 08:35:27 => *Complete, transmitted 1277 bytes in 1 second (1.25K cps)
Fri Oct 05 08:35:27 => *Upload complete, 1 file sent.

I'm trying out my regex in the new syslog parser screen and i'm running in to issues.  A single line of regex which captures all of my required groups in RegEx101.com completely fails in the SIEM.  So I broke it up in to separate lines of Regex for each line of the log that I want to look at and it captures the correct groups in the sample log field, but it only maps out the keys and values of the first line, even though the groups are highlighted in the second line.  

 

parser.jpg

 

The sad thing is, I used to support this product for almost two years ago working directly with McAfee and I never saw this happen with any of the customers I worked with.  

 

Anyone seen this before and know how to get around it?  I've already tried using \R in the Regex to denote the carriage return and see if that will get me the all the capture groups, but it looks like the SIEM doesn't utilize \R

 

Any help would be GREATLY appreciated! Thanks!

3 Replies
David1111
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 2 of 4

Re: Parsing multi line events

Hi,

Could you Share the REGEX Syntax your trying to performe !?

Thanks'

Re: Parsing multi line events

This is the RegEx broken up in to seperate lines;

 

(\w+\s\w\w\w\s\d+)\s(\d+:\d+):\d+\s+=>\s\*(Sending\sfile)\s\D(\D\D+Aps_\w+\D+\d+.\d+.\w+)\D
\w+\s\w\w\w\s\d+\s\d+:\d+:\d+\s+=>\s\D(\w+)\D\s\w+\s(\d+)\s\w+\s\w+\s\d+\s\w+\s\D+(\d+.\d+K)\s\w+\D+\
\w+\s\w+\s\d+\s\d+:\d+:\d+\s+=>\s\D\w+\s\w+\D\s\d+\s\w+\s\w+\D

And this is it as one contiguous line of RegEx

 

(\w+\s\w\w\w\s\d+)\s(\d+:\d+):\d+\s+=>\s\*(Sending\sfile)\s\D(\D\D+Aps_\w+\D+\d+.\d+.\w+)\D\R\w+\s\w\w\w\s\d+\s\d+:\d+:\d+\s+=>\s\D(\w+)\D\s\w+\s(\d+)\s\w+\s\w+\s\d+\s\w+\s\D+(\d+.\d+K)\s\w+\D+\R\w+\s\w+\s\d+\s\d+:\d+:\d+\s+=>\s\D\w+\s\w+\D\s\d+\s\w+\s\w+\D

 

I've also tried swapping out the \R's for \s's and that parses in RegEx101 as well, but also fails completely in the SIEM.  So far, the closest i've gotten is running the multiple lines of RegEx, which acts as you see in my screenshot above.

 

(Note, I did made a small change to add another capture group for the speed of the file transfer as we want to know that as well. )

David1111
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 4 of 4

Re: Parsing multi line events

Hi 

it seems that the regex is not 100% Dynamic...

Please Share a example of the row log (packet)

just don't forget to change ditailes (ip addresses, Hosts etc..)

 

Wating to see how we could helpout.

best regards.

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community