6 Replies Latest reply on Nov 13, 2014 11:57 PM by rcavey

    Best practice for ELM and their storage

    parinya.ekparinya

      Hi Guys,

       

      • If Receivers and ESM are located in different geolocation, which side should we place ELM? (near to ESM or near to Receiver?)
      • Does events ELM pull from Receivers compressed? Would it benefit from WAN optimization solution?
      • Any high availability deployment available for ELM? As far as I know, ELM can keep their log in a pair of mirror storage pool but if ELM die, however, no raw logs can flow to storage pool. I aware that Receiver can cache evetns until ELM comes back alive and pull them. Is that the only solution for now?
      • How ELM store their raw log? Are they kept in a database or just a bunch of files? Any encryption protection? If any what's a standard used?
      • For All-in-one combo & REC-ELM combo, we need to a storage device for "Full Text Indexer" (FTI). It is recommended that this be at least 20% of the space currently allocated for all ELM storage pools on the system. But since FTI bound to a storage device, what should we do if we want to add more storage pool later thus cause FTI below than 20% of all storage pools. Can we relocate FTI to another bigger storage device later? Or should we sizing it in the first place and adding storage pool later is not recommended?
      • If we can relocate FTI, what's the sizing effect during FTI relocation?

       

      That's all I can think of for now. Thank you for your effort.

       

      Best regards,

       

      Parinya

        • 1. Re: Best practice for ELM and their storage
          Chris Boldiston

          Hi Parinya

           

           

          I have put the answers to your questions below;

           

           

          • If Receivers and ESM are located in different geolocation, which side should we place ELM? (near to ESM or near to Receiver?)
          A - The ELM should be placed near the Receivers

           

          • Does events ELM pull from Receivers compressed? Would it benefit from WAN optimization solution?
          A - The Receiver compresses files and then they are periodically pushed to the ELM.

           

          • Any high availability deployment available for ELM? As far as I know, ELM can keep their log in a pair of mirror storage pool but if ELM die, however, no raw logs can flow to storage pool. I aware that Receiver can cache evetns until ELM comes back alive and pull them. Is that the only solution for now?
          A - This is our current solution. A PER can be logged at https://mcafee.acceptondemand.com/index.jsp for direct interaction with PM on adding additional functionality to the product.

           

          • How ELM store their raw log? Are they kept in a database or just a bunch of files? Any encryption protection? If any what's a standard used?
          A - There is no encryption of the data in ELM but the data is checksummed and an Integrity Check can be run to make sure it has not been modified. The ELM records are kept in a compressed file format.

           

          • For All-in-one combo & REC-ELM combo, we need to a storage device for "Full Text Indexer" (FTI). It is recommended that this be at least 20% of the space currently allocated for all ELM storage pools on the system. But since FTI bound to a storage device, what should we do if we want to add more storage pool later thus cause FTI below than 20% of all storage pools.
          A - Having less than 20% will not break the FTI it will simply reduce its effectiveness on enhancing ELM searches.

          • Can we relocate FTI to another bigger storage device later?
          A - Yes this can be moved on the ELM properties / Configuration screen where you setup the full Text Indexer.  During the move the ELM may be unavailable until all the files are successfully migrated to the new location. 

          • Or should we sizing it in the first place and adding storage pool later is not recommended?
          A - It is best to size the FTI properly the first time.  If you think you will need storage growth in the future it may be save to allocate extra percentages to FTI to avoid having to move it in the future.

           

          • If we can relocate FTI, what's the sizing effect during FTI relocation? 
          A - When FTI is moved it will build the new indexes on the new location.  The ELM will shut down while additional files are copied and the ELM is linked to the new FTI location. 

           

           


          Regards

           


          Chris

          • 2. Re: Best practice for ELM and their storage

            Chris,

             

            What if I need to make a copy of the ELM archive for permanent offline storage?  We have a need to permanently store the raw archives from the ELM to offline media, so we want to make copies of the ELM logs, but not change the ELM DB or storage pool configuration.

             

            How can we do this?  Do we just have to run a giant ELM search and export the results?

             

            Thanks,

            Greg

            • 3. Re: Best practice for ELM and their storage
              Chris Boldiston

              Hi Greg

               

               

              It sounds like you want to be able to store that raw archive off line and then bring it back online if needed or, be able to use another method to extract data from that archive? If that is what you need then I am not aware of a way to do that with ELM.

               

              The design of ELM means you can define a retention period pools and then data is kept for that amount of time. An ELM mirror can also be setup to mainitain availability of data. If you have ELM configured correctly you will have Confidentiality Integrity and Availability for your data. It will also be easily accessible.

               

              Let me know if this helps,

               

               

               

              Chris

              • 4. Re: Best practice for ELM and their storage

                Hi Chris,

                 

                We have end customers who have requirements for indefinite data retention, so we are indeed looking for a way to store the archive data offline, and then bring it back online if neded or use another method to pull data out of the archive.  We have mirrored storage devices setup to retain data for several years, but this is apparently not going to be sufficient.

                 

                Thanks for the information - I will engage with our channel manager to see what can be done as far as a PER to enable this kind of functionality.  IMHO it would be nice to have an export function from the ELM, and then a separate application that browse/search the archived data.

                 

                Thanks,

                Greg

                • 5. Re: Best practice for ELM and their storage
                  derick

                  Hi Greg and Chris

                   

                  Has anything changed with respect to the export and import of data from/to the ELM as mentioned above?

                   

                  Regards

                   

                  Derick

                  • 6. Re: Best practice for ELM and their storage
                    rcavey

                    Chris,

                     

                    Correction for Q3  -    SIEM 9.4.2 ( Just Released )  adds ELM Redundancy, yay! been waiting 8+ months for this feature.  As soon as I can repair the broken ELM mirrors we are going to upgrade to 9.4.2 to implement this across our two ELM's.  I'll probably create a new post with our experience implementing and kicking the tires.

                     

                    Cheers,

                      -B