2 Replies Latest reply on Jan 2, 2015 4:51 AM by vinaya_k

    De-Duplication of Identical Incoming Logs

    philipshauntaylor

      Hi,

       

      Does anyone know: if an Event Receiver receives two logs that are identical, does it de-duplicate the pair of logs resulting in one event to be processed, or does it aggregate the two logs forming a single event with an event count of 2?

       

      Thanks,

       

      Phil

        • 1. Re: De-Duplication of Identical Incoming Logs
          rcavey

          Phil,

           

             Yes the receiver aggregates duplicate and even similar messages typically it is based on data source and a 5 minute window I believe which happens at the receiver level and those are retrieved by the ESM on intervals.   You can poke and the GUI and docs which do decent job and explaining the aggregation settings.

           

          Aggregation it not always a plus though because let's say one field is different i.e. username can change but the event will be aggregated.  You can always search the ELM to give you all the details surrounding that event if needed.

           

          Cheers,

            -Bob

          • 2. Re: De-Duplication of Identical Incoming Logs
            vinaya_k

            Hi,

             

            When you are talking about log duplication, is it same logs coming from same datasource but from two different receivers or aggregation of events??

             

            If it's scenario 1 then McAfee treats as 2 different logs as it's coming from different receivers

             

            If it's scenario 2 i.e. aggregation then McAfee SIEM does have mechanism built in from which identical events are grouped in. When we say identical the fields McAfee SIEM uses to aggregate are Signature ID, Source IP and Destination IP. Only if these 3 fields are identical and if aggregation level is set 1 they are aggregate for certain period of time.

             

            Hope this answers your query.

             

            Regards,

             

            Vinaya.