High volume of packet captures

[petermason]

Hi All,

Over the past few months I have seen the volume of packet captures on my NSM's greatly increase, is anyone else seeing this issues?

Running the Health Check for Packet Captures/Alert is now up to 24 pcaps/alert and it should be < 1.5 pcaps/alert according to the documentation.

I've checked the DB and the iv_packetlog table has grown very large, running queries against the DB confirms the volumes.

I 'm seeing this issue on different manager pairs on both v8.2 and 8.3.

I upgraded the v8.2 manager to the latest v8.3 HF and purged the DB and continue to have the issue.

Is anyone else seeing this issue?

Thanks

Peter

[d_aloy]

Hi Peter

Yes, I've noticed this not too long ago. The size of iv_packetlog was huge compared to iv_alert and decided to go for more aggressive maintenance tasks:

• Keep up to 10M alerts
• Keep up to 30 days worth of data/app/logs
• Make sure we have weekly tune/archival/pruning tasks scheduled
• Scheduled manual purge every few weeks.

On the master policy:

• Disabled the default packet capture for specific attacks (i.e. information and low severity, medium severity with high BTP).
• Disabled forensic pcap - only use this in case of FP or specific UDS/High Sev attacks for a very limited period of time.

This seems to have done the trick as on the busiest manager I see a 2.62 packetlog/alert ratio.

You could run a query against the db to find out which are the attacks generating larger packetlogs (by size) - but this would be a resource intensive one as you will  need to join packetlogid from iv_alert and iv_packetlog tables and calculate which packetlogid is generating the more packetdata  (calculated  with length(packetdata) think?). When I ran this query on a prod NSM I left it running overnight....

Also, have you checked the policy violation alerts you get? i.e. RC4 cipher detected, SSL3 detected, ssl client hello invalid timestamp,  abnormal rdp session termination, etc? These usually generate a lot of noise, and bring 0 security intelligence.. so depending on the customer requirements it may be possible to  disable them on the master policy.

Hope it helps mate

Regards

David

[petermason]

Hi David,

I'm still looking at this but getting confused by the results. Do you know how the alerts are associated to the packet captures?

I've run some queries that show there are multiple alerts to each packetlog and I'm wondering how I can identify the actual alerts generating the most pcaps?

(This query gets me the attack with the highest packetlog count in the last 20 mins)
mysql> select p.packetlogid, at.name, a.sensorid, count(*) pktCount from iv_packetlog p,iv_alert a, iv_attack at where p.creationtime > now() - interval 20 minute and a.creationTime > now() - interval 20 minute and a.packetlogid=p.packetlogid and a.attackidref=at.id group by p.packetlogid order by pktCount DESC limit 20;
+---------------------+-----------------------------------------------------------------+----------+----------+
| packetlogid         | name                                                            | sensorid | pktCount |
+---------------------+-----------------------------------------------------------------+----------+----------+
| 6457794843635578906 | HTTP: Apache HTTPD mod_proxy_ajp Denial Of Service              |     1105 |       91 |
| 6457796930922600145 | HTTP: Netscape Web Publisher URI Buffer Overflow                |     1105 |       55 |
| 6457795505060542635 | HTTP: SQL Injection Attack Detected                             |     1105 |       36 |
| 6457794564494097706 | DNS: ISC BIND RPZ Rule Processing Vulnerability (CVE-2017-3140) |     1098 |       18 |
| 6457794843585247745 | HTTP: Overly Long POST URI in HTTP Request                      |     1105 |       12 |
| 6457795732677038538 | HTTP: URI Too Long                                              |     1105 |       10 |
| 6457799198715647507 | HTTP: Netscape Web Publisher URI Buffer Overflow                |     1105 |       10 |
| 6457799327646381925 | HTTP: Overly Long Authentication Header                         |     1098 |        8 |
| 6457795947356135677 | HTTP: Overly Long Authentication Header                         |     1098 |        8 |
| 6457795746073107019 | HTTP: Blind SQL Injection - Exploit                             |     1107 |        8 |
| 6457796325330084943 | HTTP: Overly Long Authentication Header                         |     1098 |        8 |
| 6457796321070781708 | HTTP: Ericom AccessNow Server Stack Buffer Overflow II          |     1105 |        8 |
| 6457795664166499769 | HTTP: Blind SQL Injection - Exploit                             |     1107 |        8 |
| 6457798528698574307 | HTTP: Overly Long Authentication Header                         |     1098 |        8 |
| 6457796746270359003 | HTTP: Overly Long Authentication Header                         |     1098 |        8 |
| 6457800169459972053 | HTTP: Overly Long Authentication Header                         |     1098 |        4 |
| 6457800169560646223 | HTTP: Overly Long Authentication Header                         |     1098 |        4 |
| 6457795436850429572 | HTTP: Overly Long POST URI in HTTP Request                      |     1092 |        4 |
| 6457795436900728832 | HTTP: Overly Long POST URI in HTTP Request                      |     1092 |        4 |
| 6457800178611656437 | HTTP: Websocket handshake request detected                      |     1092 |        3 |
+---------------------+-----------------------------------------------------------------+----------+----------+
20 rows in set (2 min 14.40 sec)
(Count how many alerts have this packetlogid attached to them)
mysql> select count(*) pktCount from iv_alert where packetlogid ='6457794843635578906';
+----------+
| pktCount |
+----------+
|        7 |
+----------+
1 row in set (7 min 4.63 sec)
(Show Attack Names for the 7 alerts)
mysql> select a.sensoralertuuid, a.packetlogid, a.attackidref, at.name from iv_alert a, iv_attack at where a.packetlogid ='6457794843635578906' and a.attackidref=at.id;
+---------------------+---------------------+-------------+--------------------------------------------------------------------------+
| sensoralertuuid     | packetlogid         | attackidref | name                                                                     |
+---------------------+---------------------+-------------+--------------------------------------------------------------------------+
| 2048170276814268324 | 6457794843635578906 | 0x402cf600  | HTTP: Apache HTTPD mod_proxy_ajp Denial Of Service                       |
| 2048170276814268330 | 6457794843635578906 | 0x40214900  | HTTP: Netscape Web Publisher URI Buffer Overflow                         |
| 2048170276814268356 | 6457794843635578906 | 0x4029d300  | HTTP: SQL Injection Attack Detected                                      |
| 2048170276814268357 | 6457794843635578906 | 0x4026de00  | HTTP: Cross Site Scripting -  Script Attempt Found in HTTP request V     |
| 2048170276814268358 | 6457794843635578906 | 0x4026bb00  | HTTP: Cross Site Scripting II -  Script Attempt Found in HTTP request II |
| 2048170276814268359 | 6457794843635578906 | 0x40272500  | HTTP: Cross Site Scripting -  Script Attempt Found in HTTP Request VII   |
| 2048170276814268360 | 6457794843635578906 | 0x40208a00  | HTTP: Cross Site Scripting -  Script Attempt Found in HTTP request       |
+---------------------+---------------------+-------------+--------------------------------------------------------------------------+
7 rows in set (5 min 44.47 sec)
(Count the number of packets in the packetlog)
mysql> select count(*) pktCount from iv_packetlog where packetlogid ='6457794843635578906';
+----------+
| pktCount |
+----------+
|       13 |
+----------+
1 row in set (1 hour 4 min 33.88 sec)

[mjesmer]

This is an interesting issue. The packetlog having multiple alerts tied to it appears to be normal behavior indicating that the single flow triggered multiple alerts. As for how to figure out which attack signature is causing the most pcaps,..that might not be possible because of the multiple alerts tied to a single packetlog.

My suggestion, get support involved and ask them why the Packetlog table has grown out of control.

[d_aloy]

Hi Peter

Try this - sum(length()) is represented in MB:

mysql> select at.name, sum(length(packetdata)) /1024 /1024, count(*) pktcount from iv_packetlog p,iv_alert a, iv_attack

at where p.creationtime > now() - interval 36000 minute and a.creationTime > now() - interval 36000 minute

and a.packetlogid=p.packetlogid

and a.attackidref=at.id

group by at.name order by sum(length(packetdata)) desc limit 10;

Here a sample ouput from my nsm in the lab - hence not much data as only traffic is manual attacks

Hope this helps.

Regards

David

[d_aloy]

Hey Peter

Did you ever get to the bottom of this?

Cheers

David

[petermason]

Hey David,

The packetlog tables is full of orphan packet logs due to a number of bugs with L7 data collection, manager software and sensor software. McAfee are still working on this one but have fixed some of the issues in 9.1 and are targeting the rest for 9.2.

I'm waiting for the 9.2 release due out early next year to move to 9.x

Regards

Peter Mason

[petermason]

Hi David,

The latest update on this is that it is caused by some of the correlated alerts, when the component alert is triggered the packet captures are written to the DB immediatly, but if the threshold is not reached to generate the correlated alert no alert is written to the DB and the packet logs are orphaned. Development are working on a fix but don't have an ETA for it yet.

Regards

Peter