Skip navigation
McAfee Secure sites help keep you safe from identity theft, credit card fraud, spyware, spam, viruses and online scams
1163 Views 3 Replies Latest reply: Feb 11, 2013 9:40 PM by hschupp RSS
gabezawalski Newcomer 2 posts since
Jan 10, 2013
Currently Being Moderated

Jan 10, 2013 10:23 AM

NSM - Bulk Attack Filter Unassign?

I've got a mess on my hands.  Two attack filters were created and added to everything instead of creating an ACL.  I want to clean it up and remove them from all the applied filters (both inbound and outbound..almost 10,000 assignments).  Is there a way to do this in bulk?  I can't imagine how much time would be wasted if there was not.

 

Thanks.

  • hschupp Newcomer 20 posts since
    Dec 11, 2008
    Currently Being Moderated
    2. Feb 11, 2013 9:37 PM (in response to gabezawalski)
    Re: NSM - Bulk Attack Filter Unassign?

    Here is a document I wrote up for a customer who walked into a system that they had been adding filters to for years.. and now no-one knew why most of them had been created.

    This customer primarily used I-4010 or M-4050 or larger sensors which allowed them 128K to 256K of alert filters to be applied to a sensor.

     

    Chart 1: Attack Filter capabilites by sensor

    Maximum Type

    M-8000

    M-6050

    M-4050

    M-3050

    M-2950

    M-2850

    M-2750

    M-1450

    M-1250

    Customized attacks

    100,000

    100,000

    100,000

    100,000

    100,000

    100,000

    100,000

    100,000

    20,000

    Attack filters per VIDS

    128,000

    128,000

    100,000

    100,000

    100,000

    100,000

    100,000

    40,000

    20,000

    Attack filters per Sensor

    262,144

    262,144

    262,144

    262,144

    131,072

    131,072

    131,072

    65,536

    32,768

     

    But now they were deploying M-1250's out to their satellite offices.

    They found they could join the sensors to the NSM and receive the initial configuration but could not push a configuration update to the M-1250 sensors after that.

    The error told them they were exceeding the allowed attack filter count. 

     

    (joining a sensor to the NSM has fewer 'checks' for the initial configuration solely for the purpose of trying to ensure it DOES join)

     

     

     

     

     

    Anyways - here is a detailed document on manual "Filter Management" using MySQL commands.

     

    ALERT FILTER MANAGEMENT GUIDE  - USING MYSQL COMMANDS TO CLEAN UP THE NSM DATABASE

     

    On the NSM it is extremely easy to Create and Assign Alert Filters. 

    However, removing them is NOT so simple when they are applied across 100’s or 1000’s of Signatures. 

    This guide talks about how to easily craft mysql commands to remove Alert Filter Assignments manually from NSM DB using the MySQL prompt.

     

    Changes in the 7.1 Filter Management have improved the de-assignment of Alert Filters process.. but the methods below are still valid when you are trying to bulk changes.

     

    There are two parts to this document that are interwoven throughout:

     

    1. How to perform maintenance on your Attack Filters
    2. Why you should never assign an Alert Filter to ALL signatures.

     

    The instructions below speak to both issues. 

    For the “never assign a filter to all signatures” see the write-up under the “WHY YOU SHOULD NOT (EVER!) ASSIGN FILTERS TO ALL SIGNATURES’” section in the beginning of this document below.

     

    There are several levels of how to go about cleaning up the alert filters on a sensor.

     

    1. You can delete all the Filter Assignments in one fell press of the enter key
    2. You can delete all assigned filters from an Interface, or the sensor, or the Domain or a combination of them all.
    3. You can do so by deleting a particular filter from being assigned anywhere.
    4. You can delete all filters assigned to a particular attack.

     

    It all starts with ---

     

    STOP THE NSM MANAGER AND USER INTERFACE SERVICES – THEN…

     

    Log in to the mysql command prompt so you can run the queries.

    To do so open a command prompt in the <datadrive>\mysql\bin directory.

    Then type:

     

    mysql -uroot -p                                <CR>

    Enter your 'root' password at the prompt.

    use lf                                     <CR>

    Will respond with “database changed”

     

    From there you can start running queries to see how many filter assignments there are:

     

    Select count(*) from iv_alertfilterassignment;                 <CR>

    Will return a row count which is the total # of filter assignments.

    Do NOT let this number fool you! (see next paragraph)

                                   

    WHY YOU SHOULD NOT (EVER!) ASSIGN FILTERS TO ALL SIGNATURES:

    (especially not at the domain level!)

     

    You may get a result.. from this query above  - say it comes back with 42,645 .. and think that .. "Hey!  That is far less than the maximum!  I don’t need to clean anything out!"

    Unfortunately, that is not how it compiles when preparing the configuration to a sensor.

    The assignments are read out per Domain + per Sensor + per Interface A + per Interface B + per VID

    So, for example.. if there were 8,000 filter assignments at the domain, those would multiply as follows.. (we’ll use a I-4010 sensor that has 12 interfaces… 1A/B, 2A/B, 3A/B, 4A/B, 5A/B, 6A/B

     

    8,000   Domain

    8,000   Sensor

    96,000 Interface 8,000 x 12 interfaces

    ----------

    106,000           Alert Filter Assignments for this one sensor

     


    Now you might ask yourself how difficult is it to reach this number of filter assignments? 

    Here is an example…

     

    You decide that you want to assign a filter for your vulnerability scanner so that alerts from it are not seen in the RTTA.

    • You create a filter and assign it to all the Attacks in your policy.  The All-Inclusive with Audit Policy has approximately 4,000 signatures – in each direction (Inbound & Outbound).
    • So that is 8,000 Alert filter assignments right there.
    • You assign that to the root domain because you want it to apply to all sensors in the domain.

     

    So.. as you can see from the example above.. it only takes 1 single Alert Filter.. assigned to all signatures to suddenly have 106,000 assignments applied to a sensor.

     

    And the example there was for just an I-4010.. expand that to an M-2950 … 20 interfaces…. And you can see that this can get out of hand very quickly.

     

    The above scenario is the perfect example of why not – and, in the end, it serves to negatively impact the sensor performance having to ‘consider’ these filters for every attack generated.

     

    CAVEAT:  In the current 6.1.x.x Manager and Sensor Software code - changes were made to NOT assign filters to unused (disabled) interfaces. 

    This dramatically reduces how many filters assignments are included in a sensor configuration push.

     

     

    USE ACL’s INSTEAD

    The answer to this is to USE ACLS to do the same job. 

    Two --- yep.. Just 2!  ACLS would do the same job as assigning a filter against the entire 8000 signatures!

    You would create 1 ACL to NOT INSPECT traffic between the source and destination IPs in one direction and then 1 more for the opposite direction.

    Now.. With these two rules.. You have stopped the Vulnerability Scanner from generating the alerts in the first place.  There is no need to ‘filter’ what is never created.

    And you have saved yourself 106,000 filter assignments.

     

    Note:   When a new signature set is downloaded and applied.. the new signatures that are added are NOT going to have the filter assigned to them through any magical understanding that you had the filter assigned to all the signatures previously in the database. 

     

    So, if for example, 22 new signatures were added in the new sigset.. suddenly your vulnerability scanner would be seen generating alerts for those new signatures because the filter assignments did not include them.   You would have to constantly keep updating the filter assignments with every sigset release.

     

    The ACL will not be affected by this.. so it is a Set and Forget option that performs the same function for far less work and no follow-up maintenance.

    NOTE1: BACK UP YOUR DATABASE BEFORE PERFORMING ANY OF THE FOLLOWING DELETION SCRIPTS!

     

    THE “BARE KNUCKLE – IT’S A KO!!!” SCRIPT

     

    To simply delete all alert filter assignments you can do the following command:

     

    Delete from iv_alertfilterassignment;         <CR>

    Will delete all filter assignments from the database

    There is no recovery.. it’s down for the count.

    (untruth – a restore from a config-tables or all-tables backup will bring them back)

     

    NOTE2: BACK UP YOUR DATABASE BEFORE PERFORMING THIS OR THE FOLLOWING SCRIPTS!

     

     

    THE “KID GLOVES - GENTLER HAND…” SCRIPTS

     

    However, if you want to delete only those alerts assigned to a particular sensor you can first run the following queries to give you the variables for creating some very specific queries depending on your needs.

    Run each of the following commands and copy the results to notepad.

     

    GET ALL SENSOR NAMES AND IDS:

    Select sensor_id, name from iv_sensor;      <CR>

    Will display all the sensor names and their associated “ID”

    Copy this out to notepad or a spreadsheet

     

    GET ALL SENSOR INTERFACE NAMES AND IDS:

    Select sensor_id, vids_id, name from iv_vids;        <CR>

    Will display the sensor_ID (refer to query above) and the Interface/VIDS IDs

    Useful for queries to delete alert filters at a particular sensor interface.

     

    GET ALL FILTER NAMES AND IDS:

    Select group_id, group_name from iv_alertfiltergroup;   <CR>

     

    Will display all the attack filter names and the associated group ID for that filter

     

    GET ALL DOMAIN NAMES AND IDS:

    Select subscriber_id, name from iv_subscriber;     <CR>

    Will display domain and child domain Ids that you have created

    Useful for queries to delete alert filters at a particular domain level only.

     

    We will use the variables from these queries to craft new deletion queries that only delete specific sets of filter assignments:

     

     

    Now we’ll look at how to use these commands to remove the Alert Filter Assignments. 

     

     

     

    Continued in Next Reply

     

    Message was edited by: hschupp on 2/11/13 10:37:26 PM EST
  • hschupp Newcomer 20 posts since
    Dec 11, 2008
    Currently Being Moderated
    3. Feb 11, 2013 9:40 PM (in response to gabezawalski)
    Re: NSM - Bulk Attack Filter Unassign?

    Now that you have collected all that information -- here are the scripts to use that data to start removing the excess filters from the database.

     

    For example, to delete all filters assigned to a particular sensor you would find the sensor name in the output of the first of the queries above and then type the command:

     

    DELETE ALL FILTERS ASSIGNED TO A SINGLE SENSOR:

     

                    Delete from iv_alertfilterassignment where sensor_id=1002;  <CR>

     

    Will delete all filter assignments from the database for just that one sensor.

     

    Or, how about deleting all assignments for a particular filter from the Database?

    Is great for when you want to delete a filter from the GUI but can’t find all the places it is assigned to:

     

    DELETE AN INDIVIDUAL FILTER FROM BEING ASSIGNED TO ANYTHING:

     

    Delete from iv_alertfilterassignment where group_id=301;         <CR>

     

    Will delete all filter assignments from the database for just that one filtername.

    The group_id is found in the second of the two Select queries from above.

     

    Maybe you just want to delete the queries for a child domain in order to clean up filters (or perhaps you want to delete the child domain but can’t because there are filters assigned to it).

     

    DELETE ALL FILTERS ASSIGNED TO THE ROOT DOMAIN:

     

    Delete from iv_alertfilterassignment where subscriber_id=0;      <CR>

     

    Will delete all filter assignments from the database for the root domain.

     

    DELETE ALL FILTERS ASSIGNED TO A CHILD DOMAIN:                      

     

    Delete from iv_alertfilterassignment where subscriber_id=104;  <CR>

     

    Will delete all filter assignments from the database for that child domain.

     

    You can combine these to get even more refined:

    DELETE ALL FILTERS ASSIGNED TO A SINGLE SENSOR INTERFACE IN A CHILD DOMAIN:

     

    Delete from iv_alertfilterassignment where subscriber_id=104 and sensor_id=1003 and iv_vids=213; <CR>

     

    Will delete all filter assignments from the database for that child domain for only that sensor and only on that interface.

     

    And… as mentioned in the very beginning you can delete all filter assignments to a specific signature.

    This requires looking up the Attack ID for the signature first and then plugging that into the following query:

     

    Delete from iv_alertfilterassignment where attack_id=<attack_id>

     

    For example the attack_id for the signature “HTTP: HTTP HTML PAGE NOT FOUND” attack is 0x40280200.  The command to delete any filter assignments on that attack would be:

     

    DELETE ALL FILTERS ASSIGNED TO A SINGLE ATTACK ID:

     

    Delete from iv_alertfilterassignment where attack_id=’0x40280200’;                    <CR>

    Will delete all filter assignments from the database for that child domain for only that sensor and only on that interface.

     

    Note the use of single quotes around the attack_id value. This had not been necessary in the previous queries where the value was just a decimal value.

     

     

    THE “REFEREE’S DECISION” – SUMMARY

     

    These should give you a good handle how to craft deletion queries on the iv_alertfilterassignment table to help get your filters under control.

     

    In the basest of terms – if you do not want to detect alerts between a particular source and destination

    (which is really all a filter defines) then use an ACL to tell the sensor not to inspect that traffic. 

    That just takes TWO ACL rules instead of 8000 filter assignments.  In addition, the ACL is FAR less impactful on sensor performance

     

    Always yours in support,

     

    Henry “Hank” Schupp

     

    Did I mention?:   BACK UP YOUR DATABASE BEFORE PERFORMING ANY OF THE PREVIOUS DELETION SCRIPTS!

    <grin>

More Like This

  • Retrieving data ...

Bookmarked By (0)

Legend

  • Correct Answers - 5 points
  • Helpful Answers - 3 points