Creating a Watchlist from Malc0de

Version 1

    ESM 9.5 introduced the capability to populate watchlists with data parsed from HTTP feeds.  This provides you with the ability to tap into the vast array of public threat feeds out in the world to improve your situational awareness in ESM. In this doc we'll walk through an example of pulling a feed from malc0de.com. 

     

    The Malc0de IP blacklist is updated regularly.  Below is a screenshot of the list as seen in a web browser.

     

    0.png

     

    We can see that the list has a few header lines at the top that we'll want to ignore, and then includes a simple list of IP addresses that will be very simple for us to parse out.  To  begin, create a new dynamic watchlist with the update schedule you desire.

    1.png

    Next we'll need to provide the URL for the information we'd like to parse.  Press the Connect button to ensure your ESM can reach the URL you've specified.

    2.png

    With that done, we'll need to tell the ESM how to parse out the values on that page.  We'll specify 3 header lines, to ensure that ESM skips the unwanted lines at the top of the text file.  Then we need to provide a regex that properly captures the IP address.  In this case we can use a very simple regex of \d+.\d+.\d+.\d+.  This looks for 4 series of digits (\d+) separated by any other character (.).  This regex is a little sloppy, but works in this context. 

    3.png

    We also need to include (parentheses) around the regex we want to match, as shown above.  This turns the regex into a matching group.  In the case of this particular URL, the matching group is a bit of unnecessary hassle, since there is only one value we care about on each line.  However, in other cases you might have multiple values on each line (IP address, country, URL, email address, etc)  In this case, the matching group is used to refer to specific fields on each line.  When you've properly built your regex and applied matching group delimiters, you will see values become highlighted in blue, as seen in the screenshot above.


    The final step is to tell ESM what data type this watchlist will contain.  In our case, we'll select IP Address.

    4.png

     

    Finally, if you click "Run Now" you should find that your watchlist values are populated.  You may now use your new watchlist in correlation rules, view filters, reports, and anywhere else you'd normally use a watchlist.

     

    I hope you find this useful.  What feeds have you found useful in your own environment?