All time on the ESM is stored in UTC. Sometimes a device will appear to have the correct time, but have the wrong time zone. Is the Data Source also configured with EDT on the ESM? If you do a live stream or tcpdump on the Receiver, do your logs come in with the timestamp that you expect?
tcpdump -nni eth0 -X host x.x.x.x | less
I do realize that if the timezone of the device is EDT and I set it on the Receiver as CDT there will be a skew. Event if you set it to UTC, because the source is EDT it will be off. As I have been going through this exercise I thought I understood the function of the time delta report.
What got me to ask this question was there are devices in the report that are several days of time off even after validating the time zones.
The report will tell me a device is +3000 minutes off for example. In theory you can't have a live source have more than 24 hours of time difference but that would equate to 50 hours.
The only idea I can come up with is a historical log getting loaded into the Receiver.
Is there any relationship or pattern across the devices that this happens to? Are they on the same Receiver, or vendor type or collection protocol?
After digging around a bit I have noticed that the events from sources that are ftp'ed or similar (not WMI not Syslog) into the receiver are the ones that are farthest off in time.
They are disparate devices and do not necessarily share an operating system or other common factor that I can see.
Syslog sources have been correctly by matching time zones on source and device receiver.
Windows devices have been corrected by fixing NTP settings locally on the hosts.
That sounds like something resembling a pattern. How old are the oldest logs that could be received via a method of file transfer as opposed to a streaming protocol?
That's a good catch.
If you find that there are specific set of devices "only" which are showing time delta difference, then first which I would prefer to check is, whether the logs are received "ALL TIME" from that specific data source or not. In case if they are not, then possibility is with the services (collection/parsing) not running as expected.
One little wrinkle that can make this interesting is if different services on the data source are sending independently and using different time zones. For example, one is using UTC and the other local wall clock time. If you can identify this as the cause and cannot correct it then playing with the source IP or the port the service is sending it to can allow you to differentiate the time zones on the receiver by setting them up as different data sources.
Just suggesting this because I've run into it.