cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
jamesmac
Level 10
Report Inappropriate Content
Message 1 of 5

Something is lurking in the storage pools

Jump to solution

Fellow sufferers,

 

This is a VM-12 SIEM sitting on AWS cloud, running version 11.3.0. Two on-prem receivers feeding it (one per tenant)

 

We've set the two separate ELM storage pools up to store data for 13 months. But one of them appears to be retaining logs from as long ago as January 2020. When a colleague ran ls -lt in /elm_storage/subdirectory, this was the result: 

 

snip.png

 

Anyone seen similar behaviour? Or anyone prepared to hazard a guess at why? @pbpillai @lratcliffe ?

 

Regards, James

1 Solution

Accepted Solutions
lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 4 of 5

Re: Something is lurking in the storage pools

Jump to solution
  • Anything expired is not counted in the UI as "used" but the allocation is always "allocated".
  • You can reduce the size of the allocations and/or delete them.  The ELM will move storerooms as needed to unused allocations to free up the space.
  • This would require, as above, tuning the overall size of the space allocated to the pool to the "correct" size of the pool (as in, the space needed to store data for the time period the pool is to cover) with a small buffer of extra space to allow for growth.  This will ensure the ELM overwrites the expired allocations soon after they expire.
Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

View solution in original post

4 Replies
lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 5

Re: Something is lurking in the storage pools

Jump to solution

This is normal behaviour.  When the ELM stores data into an allocation, it compresses it and puts it into a storeroom file (max size 2GB).  These files are tracked in the SR table of the management database.  When one is full and the ELM needs a new file for a storage pool, it performs a lookup in the SR table and uses the oldest expired storeroom file.

If there are no expired storeroom files, the ELM attempts to create a new storeroom file.  If it is unable to do so (due to all the allocations being full) it will be unable to insert the new data.

 

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?
jamesmac
Level 10
Report Inappropriate Content
Message 3 of 5

Re: Something is lurking in the storage pools

Jump to solution

@lratcliffe 

Thanks Luke, that's helpful. Just to clarify a couple of points:

  • do these storeroom files count towards disk space allocation?
  • if we are running short of disk space, can we persuade the system to drop the oldest storeroom files to free space up? (we only want to keep 13 months of data, not 17-18)
  • what are the implications of having personally-identifiable data on the system that we don't expect to be there?

Kind regards

 

James

lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 4 of 5

Re: Something is lurking in the storage pools

Jump to solution
  • Anything expired is not counted in the UI as "used" but the allocation is always "allocated".
  • You can reduce the size of the allocations and/or delete them.  The ELM will move storerooms as needed to unused allocations to free up the space.
  • This would require, as above, tuning the overall size of the space allocated to the pool to the "correct" size of the pool (as in, the space needed to store data for the time period the pool is to cover) with a small buffer of extra space to allow for growth.  This will ensure the ELM overwrites the expired allocations soon after they expire.
Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

View solution in original post

jamesmac
Level 10
Report Inappropriate Content
Message 5 of 5

Re: Something is lurking in the storage pools

Jump to solution
Excellent, thank you.
You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community