cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted

Looking for a way to get full packet capture (pcap) data from NSM Database

Hello Everyone,

We have developed a custom script to deal with .pcap files and do some actions with them on a regular basis. The question was how to get .pcap files from McAfee NSM database.

Having looked at the API, we have found that there's no way to export alert capture, only sensor packet capture which is a different thing.


At the same time, we were not able to get .pcaps straight from McAfee NSM database too. Query we used is:

SELECT iv_alert.SensorAlertUUID, iv_packetlog.packetData FROM iv_alert LEFT JOIN iv_packetlog ON iv_alert.packetLogId = iv_packetlog.packetLogId

The problem is that there is layer7data only presented in iv_packetlog.packetData field, not the whole capture (Ports, IP Addresses, MAC addresses etc)

But, as we can see from McAfee NSM Interface, if you go to Analysis tab -> Threat Explorer -> View Attacks, find the attack there and click on "Export" link, you will get the whole .pcap will all the needed data. How is this thing working then?


Is there any way to export full packet capture (.pcap, not just layer7 data) with one query, like we do this by clicking on a "Export capture" link in NSM Web interface?

Thank you.

9 Replies
Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 2 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Hi Sonofliberty,

The packetData field is a longblob type field, you need to figure out how to write the data back out of it.

Regards

Peter

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 3 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

And to add what Peter said, on top of being on a blob format, I believe it is encrypted as well, so you will need to figure out how to get something you can actually read.

This is doable though, as McAfee ESM (SIEM) will show you the pcap details on the ESM interface.

We have asked for a PER on this, so that the evidence report pcap can be pulled 'on demand' from 3rd party SIEMs.

Regards

David

Highlighted

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Guys,

Thank you for your kind answers.

As there's no easy way to achieve this, I've written script that pulls new packet capture information from the database and then retrieves PCAPs one-by-one via HTTP interface. Hope it will be helpful.

#!/usr/bin/python
# Script to retrieve PCAP files from McAfee database and save them to a folder with %alert_id name
# import modules
import MySQLdb
import time
import requests


# define main function
def getPCAP():
    # define main variables
    folder = "C:\PCAPS"  # folder to which we save files
    db_name = "nsmdb_02"  # NSM database name
    db_user = "root"  # NSM Database user
    db_passwd = "pass"  # NSM Database password
    nsm_host = "nsm_http"  # NSM Hostname
    nsm_login = "nsm_login"  # Login to access NSM HTTP Interface
    nsm_password = "nsm_pass"  # Password to access NSM HTTP Interface
    while True:
        connection = MySQLdb.connect(host="localhost", user=db_user, passwd=db_passwd,
                                     db=db_name)  # Connecting to database
        cursor = connection.cursor()
        cursor.execute("SELECT MAX(creationTime) from iv_packetlog;")  # getting last packet creation time
        lasttime = cursor.fetchone()  # Fetching last packet creation time
        # execute the SQL query using execute() method.
        try:
            executedBefore  # Check if loop has been executed before
        except NameError:
            print "This is a new run, so I will load PCAPs for 1 minute ago only. Executing query..."
            query = "SELECT iv_alert.sensorId, iv_alert.sensorAlertUUID, iv_packetlog.packetData \
FROM iv_alert LEFT JOIN iv_packetlog ON iv_alert.packetLogId = iv_packetlog.packetLogId WHERE iv_packetlog.creationTime > DATE_SUB(NOW() , INTERVAL 1 MINUTE) AND iv_packetlog.creationTime <= %s"
            args = (lasttime)
            cursor.execute(query, args)
            lastquerytime = lasttime
        else:
            print "Found previous execution time. Executing query starting from %s" % lastquerytime
            query = ("SELECT iv_alert.sensorId, iv_alert.sensorAlertUUID, iv_packetlog.packetData FROM iv_alert \
LEFT JOIN iv_packetlog ON iv_alert.packetLogId = iv_packetlog.packetLogId WHERE iv_packetlog.creationTime > %s AND iv_packetlog.creationTime <= %s")
            args = (lastquerytime + lasttime)
            cursor.execute(query, args)
            lastquerytime = lasttime
        executedBefore = 1
        # fetch all of the rows from the query
        data = cursor.fetchall()
        print "Connecting to NSM to obtain PCAPs..."
        s = requests.Session()  # Establishing HTTPS Session to grab PCAPs
        http_data = {"iaction": "login", "node": "", "bwVer": "999", "Login%20ID": nsm_login, "password": nsm_password}
        url = "https://" + nsm_host + "/intruvert/jsp/module/Login.jsp"
        headers = {"Host": nsm_host,
                   "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0",
                   "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                   "Accept-Language": "en-US,en;q=0.5", "Content-Type": "application/x-www-form-urlencoded",
                   "Upgrade-Insecure-Requests": "1"}
        request = s.post(url, headers=headers, data=http_data, cookies=s.cookies, verify=False)
        # print the rows to file with name as AlertID(uuid) which is unique to each message
        for row in data:
            SensorID = row[0]
            AlertID = row[1]
            PCAP_URL = "https://" + nsm_host + "/intruvert/action/AlertLogAction?userAction=getPacketCapture&sensorId={0}&sensorAlertUUID={1}&topMenuName=INVESTIGATIONX&topMenuName=INVESTIGATIONX&secondMenuName=Threat%20Explorer&thirdMenuName=Threat%20Explorer&description=Threat%20Explorer&helpId=GUID-46AF9550-083C-4331-ABE0-4634416213BD&resourceName=%2FMy%20Company%3A0%2FInvestigation%3A0&shortResourceName=%2FInvestigation%3A0&domainName=%2FMy%20Company%3A0&currentDomainName=%2FMy%20Company%3A0&domain=false&vidsId=0&sensorName=&accessRight=fullaccess&breadcrumb=%2FMy%20Company%20%3E%20Threat%20Explorer&moduleId=13&isRootDomain=true&selectedDomain=%2FMy%20Company%3A0&OWASP_CSRFTOKEN=OACG-08MX-UJOT-64AO-SNOJ-L6WR-9NQ7-EAOA&OWASP_CSRFTOKEN=OACG-08MX-UJOT-64AO-SNOJ-L6WR-9NQ7-EAOA&nsmVersion=8.3.7.52&nsmVersion=8.3.7.52&Module_id=13&csrftokenname=OWASP_CSRFTOKEN&includeChildDomains=false&extjsDebugEnable=false&localeLnStr=".format(
                SensorID, AlertID)
            PCAP = s.get(PCAP_URL, cookies=s.cookies)
            with open(folder + "\{0}.pcap".format(AlertID), "wb") as pcap_file:
                pcap_file.write(PCAP.content)
                # close the cursor object
        cursor.close()
        print "\nLast Query time set to %s" % lastquerytime
        print "\n Waiting for 10 seconds ... \n"
        time.sleep(10)


getPCAP()

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 5 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Hi all

I need to correct myself on this thread....

The blob packetdata on iv_packetlog is not encrypted....

So it can be pulled from the db, but I found out that the packetdata field does not contain L2 to L4 headers, just L7 data.

I've logged a support case and will share the full query or procedure once I have it.

Regards,

David

Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 6 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Guys...

Guess what? Anyone checked the docs for this? (not me...definitely!)

The Integaration Guide explains how to get the pcap from the db in detail:

https://kc.mcafee.com/resources/sites/MCAFEE/content/live/PRODUCT_DOCUMENTATION/26000/PD26349/en_US/...

Page 255 of the PDF or search for "Create PCAP format packet logs"

Regards

David

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 7 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Still the issue with the L2 to L4 headers remains... I am able to pull the packetdata but that seems to be L7 only, and I don't think 'creating' headers would be accepted by any law enforcement agencies...

I've asked about this and will update the thread as soon as I have more info.

BTW, the query I've been working on is this, just FYI:

# mysql -h -u -p --database lf -e "SELECT packetData FROM iv_packetlog WHERE packetLogId =6454434134835140992;"| od -Ax -t x1z -v

This is from a 'remote' linux box, so you need to make sure the user and password and systems have the correct grant privileges on the mysql db.

Regards

David

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 8 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

# mysql -h -u -p --database lf -e "SELECT packetData FROM iv_packetlog WHERE packetLogId =6454434134835140992;"| od -Ax -t x1z -v

This will return the same output as wireshark, but L2,L3 and L4 will be in hex...the integration guide has some more on this that should help.

Regards

David

Highlighted

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Just curious what benefit this has over a separate, full pcap solution?

https://www.sans.org/reading-room/whitepapers/logging/custom-full-packet-capture-system-34177

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 10 of 10

Re: Looking for a way to get full packet capture (pcap) data from NSM Database

Replying to this a bit late..but here are my two cents:

A full packet capture solution should definitely have the same attack packets you are interested in...but you will need to find them (if you don't have any references, i.e. an IPS alert), and have some budget ready for storage....

With NSP, you can configure preattack settings to 256bytes on the sensors, and forensic pcap on the policy...and this should give you enough evidence to go to a  law enforcement agency if required. The forensic pcap is biderectional so you get both client/server full conversation - not as other IPS offerings...so you should save on the time you will need to find the relevant packets, and also in storage.

Just my opinion anyway..I would like to hear from people that would recommend a full packet capture solution instead

Cheers

David

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community