cancel
Showing results for 
Search instead for 
Did you mean: 
pepelepuu
Level 10
Report Inappropriate Content
Message 1 of 17

Last Time Values more than one hour in the future

I have been getting LOTS and LOTS of alerts stating 'Last Time' values more than one hour in the future.

HEre are the steps taken so far and other specific information:

  • ALL devices(ESM & ELM) and data sources in the environment are configured for GMT, as show below in the screen shot.
  • Runinng version 9.3.2 20140408
  • Restarted the ELM, ESM and the sample datasource
  • Rebuild Tables

log.jpg

datasource timezone.jpg

16 Replies
pepelepuu
Level 10
Report Inappropriate Content
Message 2 of 17

Re: Last Time Values more than one hour in the future

Oh btw, I also opened a ticket with McAfee Platinum support..no luck there

staschler
Level 13
Report Inappropriate Content
Message 3 of 17

Re: Last Time Values more than one hour in the future

Most often this means that you have misconfigured the time zone in the configurtion of the data source.  Your SIEM devices (ESM, REC, ELM etc.) should all have their local system time set to GMT (which it sounds like you've done correctly).  The data sources should be configured with the time zone of the logs that are being received.  If the timestamp in the Cisco ASA logs is in Eastern US time, then that's what you should have in the data source config above. 

Scott

Reliable Contributor penoffd
Reliable Contributor
Report Inappropriate Content
Message 4 of 17

Re: Last Time Values more than one hour in the future

Same issue as I posted previously.  Time settings are correct for both of the devices (Cisco ASA and Imprivata appliance) and match the setting of the ELM.  I get consistent errors in the ELM logs every five minutes for both devices.

pepelepuu
Level 10
Report Inappropriate Content
Message 5 of 17

Re: Last Time Values more than one hour in the future

@Scott

Yes, I have confirmed that ALL the devices are configured for GMT. Coincidentally, all of the devices are in the same rackspace as well.

I did a show clock to confirm, when I first noticed the issue.

a# sh clock

15:41:12.067 UTC Fri Apr 25 2014

ra#

ra#

ra# sh ntp status

Clock is synchronized, stratum 2, reference is 10.40.7.30

nominal freq is 99.9984 Hz, actual freq is 99.9958 Hz, precision is 2**6

reference time is d704fd72.b3e79c36 (15:27:46.702 UTC Fri Apr 25 2014)

clock offset is -2.6682 msec, root delay is 5.28 msec

root dispersion is 21.97 msec, peer dispersion is 18.91 msec

Also worth noting, all device use the same NTP servers as well

Message was edited by: pepelepuu on 5/7/14 1:44:53 PM CDT
staschler
Level 13
Report Inappropriate Content
Message 6 of 17

Re: Last Time Values more than one hour in the future

As a next troubleshooting step, I would look hard for these mysterious logs from the future.  One way to do this is to select a view with a time-distribution panel (for example, Event Views/Distribution).  Set your time filter for "Today", or perhaps "This Week".  It's important that you don't select "Last 24 hours" or similar timeframe.  This shows events from the most recent past, but you want to see the events that are inserted in the future.

You should see a sharp drop off in the events at the current time.  If you zoom in on the Y axis, you should be able to see if there are any logs in the database with future timestamps. Drilling into these logs should allow you to get a handle on the problem.

One other point: in your message above, I note that your device has a time of 15:41 UTC, and the NTP server has a time of 15:27 UTC.  Perhaps there was a time delay between when you executed the "sh clock" and "sh ntp status" commands.  If not, then there is something very odd indeed with time settings on your network.

Scott

Reliable Contributor penoffd
Reliable Contributor
Report Inappropriate Content
Message 7 of 17

Re: Last Time Values more than one hour in the future

Scott,

I used the method you described to identify the "future" log events.  In my case they are originating from a Cisco VPN.

When I look at the details, the "first time" and "last time" values are in the future, typically three hours ahead of the ELM time.  The packet contents look like this:

<164>May 08 2014 11:01:06: %ASA-4-722051: Group <ANYCONNECT> User <greenpa> IP <166.142.254.87> IPv4 Address <172.19.1.114> IPv6 address <::> assigned to session

The time listed in the packet is correct. I can only assume that these are some sort of log event that the ELM can't interpret, but why it would "bump" the event's time ahead by three hours seems strange.

Thanks,

Dan

Highlighted
staschler
Level 13
Report Inappropriate Content
Message 8 of 17

Re: Last Time Values more than one hour in the future

It's not entirely clear to me that we are talking about things in the same way.  I will try to be very explicit; sorry if it comes across as basic.

ELM: Enterprise log manager.  The ELM manages the store of raw logs.  It does not interpret or change these logs in any way.  The ELM system time must always be GMT.  The logs will have timestamps unchanged from how the device sent them.  It is not at all uncommon for the logs to come in with timestamps that are in a different time zone from the ELM. 

ESM: Enterprise Security Manager.  The ESM manages the database of parsed logs.  The ESM *does* deal in parsed, interpreted logs.  The logs that are stored in the ESM are converted from the local time reported in the log (and reflected in the data source config) to GMT.  All logs stored in the ESM database are normalized to GMT.  When they are extracted from the database and displayed for the user, the "First Time" and "Last Time" fields are then displayed in the time zone that the user has selected as their preferred time zone (via the Options menu in the top-right corner)

In the packet above (which represents exactly what we got from the VPN), we see a timestamp of 11:01am.  What time (in GMT) was it when that log was generated/received by the Receiver (and ultimately, the ESM?) 

If you've confirmed that

a) the logs are coming in with timestamps that are definitely reporting time in GMT

b) the data source configuration shows that the data source is properly configured for GMT

c) The First time and Last time are displaying as hours in the future.

Then something very unexpected is happening.  At this point it would probably be best to give support a call and get some deeper troubleshooting assistance.

Scott

pepelepuu
Level 10
Report Inappropriate Content
Message 9 of 17

Re: Last Time Values more than one hour in the future

Scott,

Interestingly enough, the problematic datasource for me is also Cisco VPN concentrator. I have confirmed numerous times:

a) the logs are coming in with timestamps that are definitely reporting time in GMT

b) the data source configuration shows that the data source is properly configured for GMT

c) The First time and Last time are displaying as hours in the future.

I've had McAfee technician confirm that as well. I already opened a ticket with McAfee Platinum with no results. Which why I'm on here now. I just got to frustrated.

SR 4-5731085426

Reliable Contributor penoffd
Reliable Contributor
Report Inappropriate Content
Message 10 of 17

Re: Last Time Values more than one hour in the future

Exactly as we are experiencing as well.

I have a G2M set up with our local SE, and will report back our findings later today.

More McAfee Tools to Help You
  • Subscription Service Notification (SNS)
  • How-to: Endpoint Removal Tool
  • Support: Endpoint Security
  • eSupport: Policy Orchestrator
  • Community Help Hub

      New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

    • Find Forum FAQs
    • Learn How to Earn Badges
    • Ask for Help
    Go to Community Help

    Join the Community

      Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

    • Get helpful solutions from McAfee experts.
    • Stay connected to product conversations that matter to you.
    • Participate in product groups led by McAfee employees.
    Join the Community
    Join the Community