cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Former Member
Not applicable
Report Inappropriate Content
Message 1 of 10

ESM search and filter is slow

I have ESM deployed in the last 6 months, and when I tried to search my AV logs for a particular infection (e.g. contains(ransom)) by using the filer over a 4 month period, ESM runs for over 20mins and the results are still not displayed.  Is this normal for ESM (or an SIEM) to perform this badly?

9 Replies
Former Member
Not applicable
Report Inappropriate Content
Message 2 of 10

Re: ESM search and filter is slow

When doing a 'contains' search you're no longer leveraging the indexing of the ESM which leaves you doing a full table scan. From there it depends upon how much data is being searched and how many resources (CPU/RAM/DiskIO) are available.

Anecdotally, there was resistance for years to allow wildcard searching from the Global Filter for basically this reason (mismatched performance between searches).

Former Member
Not applicable
Report Inappropriate Content
Message 3 of 10

Re: ESM search and filter is slow

This just seems wrong..

Anyone knows a better way to perform such searches?

Former Member
Not applicable
Report Inappropriate Content
Message 4 of 10

Re: ESM search and filter is slow

You can create a dynamic watchlist that performs the search in the background and keeps it updated. Then you can use that watchlist as a filter in views, rules and reports.

Former Member
Not applicable
Report Inappropriate Content
Message 5 of 10

Re: ESM search and filter is slow

Would this mean the searches apply going forward?  Or i can apply this to historical logs?

Former Member
Not applicable
Report Inappropriate Content
Message 6 of 10

Re: ESM search and filter is slow

Here are the steps do set it up. It only takes a minute and I think once you see how it works it will improve your situation.

First, create a new watchlist. I used the term 'TROJAN' for this example. It doesn't need to refresh too often since your Sig-IDs tend to stay fairly static.

watchlist-search1.PNG

Then set the source to ESM Rule Names and enter your string. Regex can be used here.

watchlist-search2.PNG

Go ahead and click Run Now. It usually is pretty quick despite warnings.

watchlist-search3.PNG

Now you can click the Filter icon above the Signature ID field and select your watchlist to filter on.

watchlist-search4.PNG

Results of this example:

watchlist-search5.PNG

It's a couple of extra steps, but it's pretty quick and easy once you know what they are.

Former Member
Not applicable
Report Inappropriate Content
Message 7 of 10

Re: ESM search and filter is slow

Hi Andy,

I'm using ESM v9.5.2 and I don't see a Source tab in Watchlist. How do I get this to appear? And did you input AV Signature ID's in the Value tab? Thanks for the suggestion, I've never done it this way.

Former Member
Not applicable
Report Inappropriate Content
Message 8 of 10

Re: ESM search and filter is slow

Is it possible that you don't have permissions for the watchlist? Perhaps try as NGCP to verify or you could post a screenshot.

The way this works is that all of the events are queried for the search string. The output of that search is a list of all of the Signature IDs that matched which automatically populate the watchlist when you click Run Now.

Now you can use that watchlist as a filter and since it's an indexed list, it will come back quickly. This is in contrast to doing a brute force table scan against a large volume of events without any indexing efficiencies that the embedded database provides.

Former Member
Not applicable
Report Inappropriate Content
Message 9 of 10

Re: ESM search and filter is slow

Hi Andy,

Please see the screenshots with NGCP permissions:

Add Watchlist screenshot.pngAdd Watchlist Value Screenshot.png

Former Member
Not applicable
Report Inappropriate Content
Message 10 of 10

Re: ESM search and filter is slow

Thank for for posting the screenshot. Please notice in mine that I have the Dynamic box checked. This should show the tab for you.

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community