cancel
Showing results for 
Search instead for 
Did you mean: 
anhp
Level 7
Report Inappropriate Content
Message 1 of 16

Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Hello,

I was trying to write a custom parser for one of our applications (ManageEngine Password Manager Pro) to collect syslog and once I imported a test file, the event does show up correctly, along with like another 10 gibberish events. The regex expression I'm using for my parser is as below:

(.*?)Smiley Sad\d*.\d*.\d*.\d) ([a-zA-Z\_]*) \d*\/\d*\/\d* \d*:\d*:\d* ([a-zA-Z]*) (.*) ([a-zA-Z0-9-]*)Smiley Sad.*)Smiley Sad.*)

My sample syslog:

admin:127.0.0.1 Account_Added 2009/12/23 11:39:00 Success pmp_test windows-server1:account1:Testing


Using this regex, I was able to capture everything, except the date and time which I don't need. When I finished rolling out the parser to the data source and imported the test log file, I got the events, but also received weird gibberish like below:


Screen Shot 2016-05-16 at 2.10.51 PM.png


Does anybody know why these are being captured?


Thanks,



Anh Pham

1 Solution

Accepted Solutions
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 3 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Without looking too closely at your regex I'll say that's a lot of .*'s and ask you to try this:

(\w+)\x3a((?:\d{1,3}\x2e){3}\d{1,3})\s([^\s]+)\s(\d+\x2f\d+\x2f\d+\s\d+\x3a\d+\x3a\d+)\s(\w+)\s([^x3a]+)\x3a(\w+)\x3a(\w+)

Also make sure 'Log Unknown Syslog Event' is selected under the data source and not 'Parse as generic syslog', which should basically never be enabled, ever.

15 Replies

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

First, I've not yet tried a parser, but I do have a dosey coming up.

I might check where your parser is configured and the data sources it is applied to. Also you might take a look at the generic syslog setting in the Data source config.

Rick

McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 3 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Without looking too closely at your regex I'll say that's a lot of .*'s and ask you to try this:

(\w+)\x3a((?:\d{1,3}\x2e){3}\d{1,3})\s([^\s]+)\s(\d+\x2f\d+\x2f\d+\s\d+\x3a\d+\x3a\d+)\s(\w+)\s([^x3a]+)\x3a(\w+)\x3a(\w+)

Also make sure 'Log Unknown Syslog Event' is selected under the data source and not 'Parse as generic syslog', which should basically never be enabled, ever.

anhp
Level 7
Report Inappropriate Content
Message 4 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Thanks guys for responding!

: Having "Parse as generic syslog" on was indeed my problem. I changed it to "Log Unknown Syslog Event" and the gibberish is gone. Now that we're talking about this though, when should I have "Parse as generic syslog" enabled? I'm asking because I was working with a consultant partner with Intel and he helped me develop this parser and basically said the opposite, that I should have "Parse as generic syslog" on and should never have "Log Unknown Syslog Event" on, and when he saw the gibberish, he couldn't explain why.... so much for McAfee partner (Allen Corp)...

Also, thanks for the regex clean-up!

McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 5 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Parse generic syslog existed long before Log Unknown and really probably should have been removed when Log Unknown was added. You will not have a reason to enable Parse as generic, and now that you've had it enabled, you'll want to go into your Policy Manager and delete the autolearned rules (Data Source then under Edit on the menu).

Use Log Unknown when working on parsers. Another tip would be to have the logs collected on a syslog server until your parser is done so you can test it with a larger sample.

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Andy

Thank you for the very concise and helpful response. I've copied it off to a notepad for future reference on an upcoming parser project, Physical security badge system logs.

Rick

anhp
Level 7
Report Inappropriate Content
Message 7 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Rick,

I'm actually about to start on a parser for a physical security badge system as well, what product are you using for your badging solution?

Thanks,

AP

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

Lenel onguard, which one are you starting?

Rick

anhp
Level 7
Report Inappropriate Content
Message 9 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Rick,

Oh, that looks way better than ours for sure, ours is called DSX Access System (DSX Access Systems, Inc.). Just thought there was a slight chance of us wring the parser together haha..

Thanks,

AP

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

That would have been nice, Good luck.

Rick