cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
anhp
Level 7
Report Inappropriate Content
Message 1 of 16

Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Hello,

I was trying to write a custom parser for one of our applications (ManageEngine Password Manager Pro) to collect syslog and once I imported a test file, the event does show up correctly, along with like another 10 gibberish events. The regex expression I'm using for my parser is as below:

(.*?):(\d*.\d*.\d*.\d) ([a-zA-Z\_]*) \d*\/\d*\/\d* \d*:\d*:\d* ([a-zA-Z]*) (.*) ([a-zA-Z0-9-]*):(.*):(.*)

My sample syslog:

admin:127.0.0.1 Account_Added 2009/12/23 11:39:00 Success pmp_test windows-server1:account1:Testing


Using this regex, I was able to capture everything, except the date and time which I don't need. When I finished rolling out the parser to the data source and imported the test log file, I got the events, but also received weird gibberish like below:


Screen Shot 2016-05-16 at 2.10.51 PM.png


Does anybody know why these are being captured?


Thanks,



Anh Pham

1 Solution

Accepted Solutions
andy777
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 3 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Without looking too closely at your regex I'll say that's a lot of .*'s and ask you to try this:

(\w+)\x3a((?:\d{1,3}\x2e){3}\d{1,3})\s([^\s]+)\s(\d+\x2f\d+\x2f\d+\s\d+\x3a\d+\x3a\d+)\s(\w+)\s([^x3a]+)\x3a(\w+)\x3a(\w+)

Also make sure 'Log Unknown Syslog Event' is selected under the data source and not 'Parse as generic syslog', which should basically never be enabled, ever.

View solution in original post

15 Replies
itgfcsys
Level 9
Report Inappropriate Content
Message 2 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

First, I've not yet tried a parser, but I do have a dosey coming up.

I might check where your parser is configured and the data sources it is applied to. Also you might take a look at the generic syslog setting in the Data source config.

Rick

andy777
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 3 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Without looking too closely at your regex I'll say that's a lot of .*'s and ask you to try this:

(\w+)\x3a((?:\d{1,3}\x2e){3}\d{1,3})\s([^\s]+)\s(\d+\x2f\d+\x2f\d+\s\d+\x3a\d+\x3a\d+)\s(\w+)\s([^x3a]+)\x3a(\w+)\x3a(\w+)

Also make sure 'Log Unknown Syslog Event' is selected under the data source and not 'Parse as generic syslog', which should basically never be enabled, ever.

View solution in original post

anhp
Level 7
Report Inappropriate Content
Message 4 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Thanks guys for responding!

: Having "Parse as generic syslog" on was indeed my problem. I changed it to "Log Unknown Syslog Event" and the gibberish is gone. Now that we're talking about this though, when should I have "Parse as generic syslog" enabled? I'm asking because I was working with a consultant partner with Intel and he helped me develop this parser and basically said the opposite, that I should have "Parse as generic syslog" on and should never have "Log Unknown Syslog Event" on, and when he saw the gibberish, he couldn't explain why.... so much for McAfee partner (Allen Corp)...

Also, thanks for the regex clean-up!

andy777
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 5 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Parse generic syslog existed long before Log Unknown and really probably should have been removed when Log Unknown was added. You will not have a reason to enable Parse as generic, and now that you've had it enabled, you'll want to go into your Policy Manager and delete the autolearned rules (Data Source then under Edit on the menu).

Use Log Unknown when working on parsers. Another tip would be to have the logs collected on a syslog server until your parser is done so you can test it with a larger sample.

itgfcsys
Level 9
Report Inappropriate Content
Message 6 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Andy

Thank you for the very concise and helpful response. I've copied it off to a notepad for future reference on an upcoming parser project, Physical security badge system logs.

Rick

anhp
Level 7
Report Inappropriate Content
Message 7 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Rick,

I'm actually about to start on a parser for a physical security badge system as well, what product are you using for your badging solution?

Thanks,

AP

itgfcsys
Level 9
Report Inappropriate Content
Message 8 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

Lenel onguard, which one are you starting?

Rick

anhp
Level 7
Report Inappropriate Content
Message 9 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

Rick,

Oh, that looks way better than ours for sure, ours is called DSX Access System (DSX Access Systems, Inc.). Just thought there was a slight chance of us wring the parser together haha..

Thanks,

AP

itgfcsys
Level 9
Report Inappropriate Content
Message 10 of 16

Re: Weird syslog parsing issue causing gibberish events to show up

Jump to solution

AP

That would have been nice, Good luck.

Rick

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community