*12/30/16 - .098 - Updated to support children/client data sources.

 

We often work in highly heterogeneous environments and it's hard to predict what glue might be required to implement a use case. Sometimes you will have 95% of what you need but the last 5% needs to be customized for the environment. The API can sometimes fill that gap. In this example the need was more operational. The request was to send a report for data sources that had not received an event in the past hour. There is a similar type of output in the UI, but there's no way to export that automatically. This script is able to query the ESM, get a list of data sources and then query each one for the defined time frame.

 

While it's possible that someone has an identical need for a script like this and this fits their requirements exactly, it's improbable. My though is this might help someone looking for some examples or sample code for their own ESM API applications.

 

Briefly, this has examples of:

- Authenticating to the ESM

- Two different ways to query the ESM:

     - One returns immediate results from a command.

     - One returns a query ID that needs to be monitored and then asked again for the results

  - Yielding a datasource if there is some action or report that needs to be considered in a per data source context.

  - Example query dictionary and JSON string.

 

Working forward from this point it's not too challenging to send the data as syslog, an email or funnel it into another API call to ServiceNow. Some of my other examples include these scenarios and I'll cover them in a future post.

 

The script files can be downloaded here or cloned from : GitHub - andywalden/esm-check-ds: Check ESM data sources for inactivity.

 

The script requires Python 3 and the Requests module. It's always recommended that you set up an environment via virtualenv or conda when adding libraries or working with different versions of Python.

The script also requires that the config.ini be configured with the credentials to connect to the ESM. It's always recommended that strict permissions be set for any file containing credentials also. Just drop the logging file in the same directory also.

 

When you run the script, it will output to both a file and the screen. The file is overwritten each time the script runs:

$ python esmcheckds.py 

2016-09-24 16:46:12,824 | WARNING | Data Source has not seen any events in the past LAST_HOUR: Bro 4, 10.10.20.2

2016-09-24 16:46:13,354 | WARNING | Data Source has not seen any events in the past LAST_HOUR: cloud, 10.10.22.202

2016-09-24 16:46:14,399 | WARNING | Data Source has not seen any events in the past LAST_HOUR: mad-pc, 10.10.22.35

2016-09-24 16:46:14,977 | WARNING | Data Source has not seen any events in the past LAST_HOUR: Monster, 10.10.22.50

2016-09-24 16:46:15,523 | WARNING | Data Source has not seen any events in the past LAST_HOUR: mwg, 10.10.22.248

You can adjust the time frame and logging level using flags:

$ python esmcheckds.py -h

usage: esmcheckds.py [-h] [-t] [-v] [-l] [-c]

 

Query for inactive McAfee ESM data sources

 

optional arguments:

   -h, --help      show this help message and exit

   -t , --time     Set time frame to check for events. Default: LAST_HOUR

   -v, --version   Show version

   -l , --level    Logging output level. Default: warning

   -c , --config   Path to config file. Default: config.ini

 

Feel free to post if there are any hiccups.