0 Replies Latest reply on May 8, 2015 3:19 AM by ksudki

    ESM - Data allocation (flows vs events)

    ksudki

      Dear Community,

       

      Just wanted to share something related to the amount of data available on the ESM for events and flows as I think it could be useful for others


      In our environnement, the ESM is receiving both events and flows and that's why we set initially the data allocation to 50% for each.

       

      Not long ago, we noticed that the amount of events available in the ESM was pretty short, approx. 2.5 months whereas the amount of flows was nearly 1 year.

       

      At first, we suspected that the esm was running out of space, but after checking it was only 50%. We decided to monitor it for several days and it did not move above or under 50%.

       

      We checked with our provider and McAfee support and it appears that the ESM is pre reserving 50% of the disk space for the events and 50% for the flows.

      Which means that in case your are going further than the limit defined for the events (more than 50%) the ESM will automatically remove older events (FIFO) even if there is plenty of space on the device.

       

      To remediate or avoid this side effect I would recommend that you follow this procedure:

      • Use the flow distribution view with time period set to all
        •   Write down the time of the first flow
        •   Write down the total of flows
      • Use the flow distribution view with time period set to one month
        • Write down the total of flows
      • Use the eventdistribution view with time period set to all
        • Write down the time of the first event
        • Write down the total of events
      • Use the event distribution view with time period set to one month
        • Write down the total of events

       

      Identify which type has a the nearest time for the first event/flow -> this means that it is certainly capped to the maximum data allocation size you defined.

       

      Do some mathematics:

      - Calculate the amount of events and flows you should keep for the desired period.

      - Calculate the percentage for each events and flows based on the total you calculated previously

      *I assumed that an event was consuming approx. 20% more than a flow, so I added an extra 20% to the events.

       

      Change the value accordingly. (be careful changing this could erase some data.)

       

      Feedbacks are more than welcome !

       

      Best regards