Regex in a Dynamic Watchlist

[siemple]

I have been attempting to build a watchlist with some test regex for a use case.  Ultimately, we'd like to use a watchlist with a regex string to detect certain variable URL patterns normally associated with malicious traffic.  For many of these threats, regex is readily available from security sites.

I've created a test watchlist and an accompanying alarm.  I've yet to have a succesful hit, so I'm attempting to troubleshoot.  When reading the ESM documentation covering watchlists, I found the following:

6.	Select the type of data this watchlist will be watching for by clicking on the down arrow in the Type field.

  When searching by string, the search will not be filtered by the type selected; all matching strings will be returned. Specifying a type simply assigns the search results to a field type, allowing the watchlist to be used throughout the system (i.e.,  filters or alarms).

I believe that this means that the regex is run against the entire packet, should there be a positive match it will then so a positive result for the alarm that triggers off a "field match."  Is that correct?  My fear is that the regex is actually only being run on the "Type" field on the event, and if that's the case, it limits the usefulness of regex watchlisting to only those fields.

Is there possibly an example of building a dynamic watchlist that I can adapt for our needs?

Also, is there a character limit to the "search:" field?

Message was edited by: siemple on 6/27/13 4:53:18 PM CDT

[rhinomike]

Would you know the answer for this question?

[staschler · Accepted solution]

Regex in a dynamic watchlist is not run against the packet.  It's run against the string table, which includes all parsed data elements of type "string".  A subtle difference, but important. 

the regex search will return all matching strings (regardless of type), and place them in the list.  The watchlist has a data type associated to it...in a sense you will be typecasting all the matching strings to the type you've defined for your Watchlist.