I have a data source that monitors web events, and have written a correlation rule that says whenever one of a list of 10 or so event types are found on this device, create a correlation. I then have an alarm set to fire whenever the correlation rule event is seen by the ESM.
Earlier today the web monitor data source made a hit at about 1130 hours. The device sent an email to my SOC team who dealt with the issue. Then, at about 1500 hours, the ESM alarm fired. I'm trying to troubleshoot the reason for the delay. This is new behavior; the correlation and data source have been working well for about 15 months.
This turned out to be a problem with the ACE. I am running 9.3.2 and evidently there is a known intermittent issue with the ACE java process sometimes hanging. My ACE was slow on Monday, then stopped processing correlations altogether on Tuesday. On Wednesday I restarted it and everything started working as normal.