cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
gilevi
Level 8
Report Inappropriate Content
Message 1 of 4

Taking the old rows from db table

Jump to solution

Hello,

I'm trying to take the old rows from db table.

for now, the collector just taking the new rows and send it to the siem but i need the olds too, at least one time.

the exist xml file (the siemcollector config file) query is : 

<Query>SELECT [dbo].[table].[action],       from table  etc...

I want to change it to :

<Query>SELECT top (5000) [dbo].[table].[action],  from table

when Im trying to change it manually it is getting overide by the siemcollector settings again and again.

 

any ideas?

 

thanks

1 Solution

Accepted Solutions
lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 4

Re: Taking the old rows from db table

Jump to solution

Change the bookmark instead of the query.  The bookmark file is in <siemcollectorinstalldirectory>/plugins/<pluginid>/

It's a plain text file and contains a value which matches your selected bookmark field.  Stop the SIEM Collector service, edit this file and start the service again.

You will receive duplicate events - i.e. anything that you have already collected will be collected a second time as SIEM Collector reads forwards through your database.  The volume of data retrieved could cause performance issues on the database, the system running SIEM Collector, your Receiver, ACE, ELM and ESM.  Due to this try to limit the retrieval of old data to the minimum amount possible.

Also check for the setting that Restricts insertion of Historical Data - on SIEM 10.x this is in the ESM properties under the Database tab and on 11.x it is in the Receiver properties on the Events, Flows and Logs tab.  If historical insertion is restricted, it is likely that this older data will never make it to your ESM.

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

View solution in original post

3 Replies
lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 4

Re: Taking the old rows from db table

Jump to solution

Change the bookmark instead of the query.  The bookmark file is in <siemcollectorinstalldirectory>/plugins/<pluginid>/

It's a plain text file and contains a value which matches your selected bookmark field.  Stop the SIEM Collector service, edit this file and start the service again.

You will receive duplicate events - i.e. anything that you have already collected will be collected a second time as SIEM Collector reads forwards through your database.  The volume of data retrieved could cause performance issues on the database, the system running SIEM Collector, your Receiver, ACE, ELM and ESM.  Due to this try to limit the retrieval of old data to the minimum amount possible.

Also check for the setting that Restricts insertion of Historical Data - on SIEM 10.x this is in the ESM properties under the Database tab and on 11.x it is in the Receiver properties on the Events, Flows and Logs tab.  If historical insertion is restricted, it is likely that this older data will never make it to your ESM.

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

View solution in original post

gilevi
Level 8
Report Inappropriate Content
Message 3 of 4

Re: Taking the old rows from db table

Jump to solution

Thanks!!

itzikn
Level 8
Report Inappropriate Content
Message 4 of 4

Re: Taking the old rows from db table

Jump to solution

i'm trying to collect rows from large db table but see the bookmark file value forwarding by 5000 once in 5 minutes, the rows on this table fills up faster then the collector collect and the siem is not at a pace can i do somthing to change it for collecting more then 5000 in 5 minutes?

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community