8 Replies Latest reply on Nov 8, 2017 4:01 PM by akerr

    Data is not available in Raw packets and ELM Archive fields

    davidp64

      Hello Members,

       

      We have noticed that for some of the events Raw data packets is not available in Raw packets fields and in ELM Archive fields too.

       

      In Raw packets there is a message i.e. Data packet is not available

      In ELM Archive there is a message i.e. Unable to retrieve ELM archive.The log have not been sent to the ELM yet.

       

      Need to understand the below queries:

       

      1.What is the meaning of this information---Unable to retrieve ELM archive.The log have not been sent to the ELM yet.

      2. Without any information in these two fields than how the data are normalising in details and custom fields..

      3. Up to what size of data/Packet is going to send to ELM..

       

      Cheers..

        • 1. Re: Data is not available in Raw packets and ELM Archive fields
          sssyyy

          How long has it been since event is parsed into ESM from ERC? 5MB or 4 hours old, ERC send to ELM.

          • 2. Re: Data is not available in Raw packets and ELM Archive fields
            minki

            does your receivers sending data to ELM ? You can verify it by running tcpdump at receivers but putting the elm IP.

            • 3. Re: Data is not available in Raw packets and ELM Archive fields
              davidp64

              Yes the receiver is sending data to ELM.

              • 4. Re: Data is not available in Raw packets and ELM Archive fields
                minki

                Dav, if that is the case then I can possibly think of -

                If the logging is enabled for that particular data source for which you are trying to fetch the raw data ?

                Try this

                1> In UI select a receiver>data source>select any event> and then in dashboard  for that particular data select an event.

                Then from the Event Summary>Event drildown>Events --> copy something like user name,SIP or DIP etch then again from the Event Summary page>Select the Search ELM and try to search whatever you selected from the event.Then you should see some results if not then something could wrong with the configuration of pools.

                2> In Under Receiver Properties>Sync ELM

                Are there any raw logs from other devices for this particular receiver ?

                • 5. Re: Data is not available in Raw packets and ELM Archive fields
                  davidp64

                  Thnks minki.

                   

                  For 1>..Is it helps because if we perform ELM Search operation on ELM w.r.t to SIP, DSIP than it will run a query to ELM and check the availability of the request data, It is not going to check w.r.t particular Data source. So it is difficult to pickup those info which are unique for problematic data source.

                   

                  We are not facing this issue for all the Data sources, there are some of the data source whose raw packets and ELM archive field is empty.

                  That's the reason i m try to understand this msg Unable to retrieve ELM archive.The log have not been sent to the ELM yet.

                   

                  Also checked sync ELM but no luck.

                   

                  Great Day

                  • 6. Re: Data is not available in Raw packets and ELM Archive fields
                    minki

                    If you able to see data on ELM for other devices from this particular receiver then I don’t see any communication issue but still can you please try this.

                    1> SSH ELM from receivers

                    2> run this command on receivers and check if the count is decreasing -

                    cd /var/log/data/inline/thirdparty.logs/elm.logs/watch -d 'ls | wc -l'


                    Let me know if it looks fine.

                    • 7. Re: Data is not available in Raw packets and ELM Archive fields
                      rth67

                      The RAW Packet information will only be available as long as the data resides on the Device (Receiver, ACE, APM, DEM), if it is a busy Receiver, the data will roll off as space is needed to accommodate newer events, you would then have to retirieve the logs from the ELM. Hopefully you enabled ELM logging, and the storage pool retains the data for a long enough period of time for you to fulfill your compliance / retention limits.

                       

                      For example: A low volume DMZ Event Receiver might have data locally back for a year, whereas a primary data center Receiver may only be able to keep 1 month's worth of data.

                       

                      If you are getting a failure to retrieve from the ELM, if it was a recent event, as someone previously stated, the data will only be written in to the ELM database once it reaches a particular threshold per data source.

                      • 8. Re: Data is not available in Raw packets and ELM Archive fields
                        akerr

                        Couple of quick questions:

                         

                        First, what version are you running?

                        Second, what is the Device Record ID  you're at (it's in the advanced details tab)?

                         

                         

                        Here's my thoughts, and what I have personally have run into. 

                        In version previous to 10.1 (might be previous to 10.1.2 - can't remember off hand), there's actually an integer overflow in the command that pulls the raw data from the ELM.  So if you're over a certain value,( I think it's 9,223,372,036,854,775,807) you can't pull data this way until you upgrade to 10.1.x.  There is a workaround from the CLI I found, but you're best option is to upgrade if this is the situation.