Showing results for 
Search instead for 
Did you mean: 

McAfee IPS - How to write a custom signature (UDS)



     The purpose of this document is to illustrate the process of creating a custom UDS or user-defined signature in NSP (Network Security Platform).  The example is meant to be simplistic designed to show the workflows and provide an easy to follow example.

          *Note - Writing your own signatures can be very powerful but a poorly written signature can have a performance impact on your sensor.  Following the template and workflows in this "how to" should help you avoid many of the potential pit-falls that exist in creating a custom signature. ALWAYS TEST CUSTOM SIGNATURES IN A TEST ENVIRONMENT PRIOR TO BROAD ADOPTION.





      In this environment the Network Security Manager was running version, the sensors are on, and the IPS Signature set is


     Identifying the block trigger

               This step may or may be useful to everyone, however, the reason for this step is to show the logic and the steps used to create the final signature.

               After Identifying a program that was outside of the scope of the corporate use policy I used Wireshark to identify a string I would like to block.




               In this case, my goal is to block a specific application from being downloaded in my network.  So using Wireshark I identify the specific string that points to the file on a remote webserver. Now that I have this information I can use it to create a custom signature

     Creating a custom signature on the Network Security Manager


               Open a browser and navigate to "Policy > Advanced > Custom Attacks".  Here you will see a list of different attacks or signatures previous imported or created.  Click on the "Custom Attack Editor" button on the right hand side.


custom attack editor.JPG


               A window will open titled "Custom Attack Editor".

               To write a new signature go to "Attack > New > McAfee Attack Definition > Exploit Attack" to open the attack editor template


New Attack editor.JPG


               You'll immediately notice that there are two options for creating a new attack definition.  The first option is to use a wizard that will allow you to write all aspects of your signature.  The second option and the one used in our example is to use a template.  Since we identified a URL that we would like to block our option is the first in the list.  However, you can also choose to detect an e-mail file name, a DNS query or response, a specific string or TCP connection from a specific IP address.




               In our example select "Detect a URL" and then select "Next" on the bottom of the page.

               The next window is the "General" windows, this is used to define the attributes of your custom attack. The fields that need to be filled out are:




               Protection Category:

               Blocking Logic:




Create Attack Definition.JPG   


               After filling out the variables click "finish".  This will return you to the "Custom Attack Editor" where you'll notice the new rule. In the "NSP Attack ID" column instead of an "ID" there will be a "Pending".  In order to add this attack to the IPS Policies, the custom attack needs to be saved.  This is done by going to "File > Save" this will update the signature set for ALL existing policies.

     Verify and edit the new signature

               At this point, you could deploy pending changes and hope that everything is working correctly.  I like to visually verify that all the changes I had hoped to make have been applied and verify their response.  To do this I navigate to "Policy > Intrusion Prevention > IPS Policies".  Here select one of the policies you have deployed on an interface.  In this case, I have selected the "Default Inline IPS" policy.  Double click this policy.


IPS Policies.JPG


               On the "Attack Definitions" tab in the "Default Inline IPS" window apply a quick filter on in the "Attack Name" field. In our example since my Attack name is "Password_Cracker" I type in "cracker" then select "apply" on the right-hand side of the window.  In the lower window, all the signatures are filtered to include only those with the work "cracker' in the attack name.


cracker filter.JPG


               To edit the response double click "UDS-Password_Cracker".  This will open the "Attack Detail" window.  By default, this signature has disabled blocking.  I'd like to change that behavior to block access to the intended URL.  To do this find the "Blocking Option" toward the bottom of the windows and select the "Customize Blocking Setting" then select "Enable Blocking"


Enable Blocking.JPG


               Select "OK" at the bottom of the page to return to the "Attack Definitions" tab.  If this is a bi-directional rule you'll need to apply the rule response to "Outbound" as well.  When that is finished click "Save" which is now RED.  This will bring you to a summary page where you will see the changes made.

     Deploy pending changes

               At this point, the rule now exists in all your policies on the network security manager and a specific response has been customized based on the ports you are inspecting.  However, the change HAS NOT been applied to the sensor.  In Order apply these changes to the sensor you need to "Deploy the Pending Changes". In order to do this navigate to "Devices > Global > Deploy Pending Changes" and select Deploy.


Deploy Pending Changes.JPG


               *NOTE - if there are additional changes waiting to be deployed please select only the ones you want to apply.

     Testing the signature

               In a test environment try to download the application from the blocked URL with your Real-Time Threat Analyzer open to view the alert response.  In this case you can see that the page was blocked and alerted.

Threat Analyzer.JPG



               Writing custom signatures can be very useful but should also be done carefully and in a test environment.

Additional Resources

               Product page:

               Product downloads/upgrades:

               Sales information:

               Data sheet:

               Snort to McAfee Network Security Platform:


This is very helpful. Perhaps can be updated for NSM 8.3x.

Version history
Revision #:
3 of 3
Last update:
‎03-29-2018 08:39 AM
Updated by:

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community