cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted

Max datasource rule count exceede

   Hi Guys

I am pretty new on this SIEM, and Im dealing with one problem that I cant solve by my self.

I have added a new datasource in the Local Receiver,  but it does show only one event  !Max datasource rule count exceeded", I have seen many webpages and guides but I cant find anything usefull.

Does anyone have an idea on what is going on?

Thanks a lot

8 Replies
Highlighted

Re: Max datasource rule count exceede

It sounds like you have configured your data source with the option to "Support Generic Syslog" enabled.  This is rarely desireable, and can result in what you're seeing.  What's happening is that the Receiver is looking at the first few bytes (16 or 32...don't recall offhand) and using those as a unique identifier for a Data Source rule.  For many syslog streams we receive, the first few bytes always include a unique timestamp, so you get a new DS rule for every event, quickly filling up the DS rule table.  As new events come in, the Receiver tries to create new DS rules, but cannot, so it generates this error.

You will need to disable this option on all of your data sources.  Then you will need to delete the auto-leard DS rules via the Policy Editor.  This will clear up the error condition you are currently seeing.  Then you will be able to begin troubleshooting why your events coming in from the new data source.

Scott

Highlighted

Re: Max datasource rule count exceede

Hi Scott

Thanks a lot for your reply and my apologies for my delayed reply.

Context: Iḿ trying to integrate Nitro with a non out-the-box appliance, therefore I am using the Generic option in the add datasource page.

I have tried to follow your instructions, but I had some questions;

1)  Disable Support Generic Syslog

-> Does it mean that I have to set the Supprt  "Do nothing"? Should I use the profiles? (in this case there is no datasource model associated and it does not allow me to create the datasource.

2) Policy Editor

I had clean the auto-learned rules.

Thanks in Advance Scott

Highlighted

Re: Max datasource rule count exceede

As Scott mentioned, the parse as generic syslog option is good for troubleshooting, but can create an excessive amount of data source rules resulting in the policy violation you witnessed. Adding a data source in this manner will bring events into the ESM, however the rule name will be the first several bytes of the message, and the source IP is set to the IP of the data source. Other useful fields such as a source user, destination IP, etc. are not parsed out, and are therefore not searchable as fields in the ESM.

The best option is to turn on the generic option for a short period of time to gather unparsed events and then create a new Advanced Syslog Parser rule or rules to properly parse the data. In the interim, simply set the option to "Log Unknown". This will group all unparseable events into a single aggregated event, making you aware that events haven't parsed, while not overloading the system with generic rules that provide little value beyond troubleshooting. Your ultimate goal should be the creation of custom ASP rules to parse the data in such a way that it is useful in your reports and searches.

Mike

Highlighted

Re: Max datasource rule count exceede

Hi Mike, thanks for answering...

I´ve created a ASP rule for my out-of-the-box appliance syslog messages  that fits perfectly with what I see in the Data source rule when I have Support Generic Syslog activated, but when I unchek this option, there is not event at all in my dashboard, probably because it do not match correctly. any idea?

How I can say to the SIEM that an specific ASP Rules is attached to specific device? is that possible?

test3.png

Highlighted

Re: Max datasource rule count exceede

You will need to make sure the ASP rule is enabled for the policy for that device. You can find this by going to the policy editor, and to the ASP rules. Click the arrow by default policy and select your data source. Make sure the rule is enabled for that data source. It is probably disabled at that level but enabled for the default policy. You need to enable it at the data source ASP policy level and this should help. In the first image notice the highlighted rule is disabled for the DLP Policy Discover Rule. In the second image it is enabled but I am scoped to the device ASP policy. This is what you want to verify and probably enable for your parser.

Default policy.JPG  Policy device level.JPG

Highlighted

Re: Max datasource rule count exceede

Hi, thanks!

I have followed your indications, however I´m still not sure whether I´m associating the ASP rule with the device.  When I goes to the Policy editor of the device, the advanced filter puts the device id in it, so I just look for my ASP Rule called ASP, but nothing appears, just when I delete the Device type ID is when it appears...

I have even create another rule when I had the device selected in the Policy Tree...there is another way more straigh foward to associate? it´s driving me crazy....;-)

test4.png

Highlighted

Re: Max datasource rule count exceede

Yes, I met this issue too. It looks like a huge bug on Nitro..

1) Create a new log source as a Generic Data Source

2) create 5 new ASPs associated with that specific data sources..

3) I can see events coming with a bunch of unknown events..

4) I drilled down few "unknown events" -> pasted the raw packets to the parser and the parse proved to be working fine with the logs.

that means the ASP module does not parse the events properly...

Regards

wilson

Message was edited by: wilson.wang on 4/14/14 10:12:08 PM CDT
Highlighted

Re: Max datasource rule count exceede

If you have already created custom ASP rules to parse the events, go to the rule editor and try checking the box, "Include syslog header in regular expression match". ASP attempts to match and parse the syslog header automatically. By default it is assumed that your regular expression is meant to match on the data after the header. If you check the box, then your expression will be evaluated against the entire event. This could be why your expressions match in the rule editor, but not in running policy. Also check the content strings you have defined, as the rule will not fire if the content strings don't actually appear in the log.

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community