Showing results for 
Show  only  | Search instead for 
Did you mean: 
Level 10
Report Inappropriate Content
Message 1 of 4

A HIPS custom expert rule success story using Buffer Overflow, Caller_Module and target_bytes (undocumented stuff)

After a long struggle with HIPS and determining that the basic custom file name/path, reg key, service names etc, and especially Default rules are not high fidelity enough to block on (and sometimes log on) in a large environment, I've finally found something that is worthy!

This rule uses some undocumented keywords in an expert rule to finely tune in ways the GUI and exception rules cannot.

Enter this rule:

Rule {

                Class "Buffer_Overflow"

                tag "target_bytes test"

                Id 4303

                level 3

                application {Include "*"}

                Caller_Module {Include { -path "*\\BadPath\\*"} }

                target_bytes { Include {04 f9 ?? ?? 00 20 68 ?? ?? ?? ?? e9 ?? 13 e2 5f-?? ?? 3c ?? ?? e9 ?? 42 ?? ?? 68 ?? ?? ?? ?? cf} }

                dependencies -d -c "432" "434"

                directives -c -d "bo:call_not_found"


(variables have been changed)

Let's Break it down:

1. At its core, this rule is a copy of rule 6013 "Suspicious Invocation - call not found". Remove Caller_Module and target_bytes lines and you've got it. This rule watches for an application being overflowed, and also checks if "call not found" was present in the scenario. This is one of the many buffer overflow sub categories/default rules and will eventually provide the ability to fine tune where the GUI cannot. Why can't we block this rule? Because it catches WAY too much stuff in a large environment and is almost impossible to tune by whitelisting etc. With a little luck and host analysis we can determine that some of these dll's are indeed bad. That is key.

2. You've determined one of the dll's (caller path/module) overflowing some SourceProcessNames (explorer.exe etc) in some alerts from the above rule is indeed bad. Good. But there are HUNDREDS of variations of them across your network! (Bad123.dll, bad312.dll, bad543.dll, with varying sizes).  File names and even md5's won't work here, unless you want to list the hundreds of filename variations which would also need to be udpated all the time! Luckily for us they all seem to be dropped to the same directory \BadPath\*. Awesome! Let's INCLUDE that directory instead of including EVERY caller path/module overflowing something like the default 6013 rule. This is the undocumented Caller_Module line you see above. Yes, you can "exclude" instead if that fits your scenario as well.

3. Crap! There are some good (or at least unknown) caller path/modules/dll's causing some overflows in that same \BadPath\* directory as well. We can't block them all yet! Enter (the also undocumented) target_bytes line. target_bytes is the set of 32 bytes that is causing the overflow (I think!). Hmmm... same malware creators? Same overflow code?? In this case... yes! They share at least 50% of the same code and start with the amount everytime! (see the rule above). We can INCLUDE a set of target bytes and WILDCARD (??) the pieces that seem to change from variation to variation. Yes you can exclude here as well, if it makes sense to.

What are you left with? A rule that looks for caller paths/modules/dll's (like bad123.dll) overflowing SourceProcessNames (like explorer.exe) with the "call not found" thing happening. The caller paths must also live in \BadPath\*, and finally the target bytes creating the overflow must match what you see above on the target_bytes line. Block that bad stuff! Some of you may be asking why even have the Caller_Module line in now that we know the target_bytes? Good question. I haven't gotten this far yet to determine how high fidelity target_bytes is.

Hope this helps people out there who cannot blindly block with rules, and where exception rules are not good enough or painful to constantly update.

Message was edited by: shakira on 2/7/14 9:41:44 AM CST

Message was edited by: shakira on 2/12/14 11:17:31 AM CST

Message was edited by: shakira on 2/12/14 11:17:59 AM CST
3 Replies
Level 9
Report Inappropriate Content
Message 2 of 4

Re: A HIPS custom expert rule success story using Buffer Overflow, Caller_Module and target_bytes (undocumented stuff)

I'd like to get McAfee's response on the "undocumented" areas, specifically if your assumption regarding the target_bytes is correct.


Level 10
Report Inappropriate Content
Message 3 of 4

Re: A HIPS custom expert rule success story using Buffer Overflow, Caller_Module and target_bytes (undocumented stuff)

This was their official response on what target_bytes was:

"Target bytes are the instruction code before and after the API call in shell code."

I'm assuming the first half before the dash is the "before", and after the dash is "after";

00 02 90 8b c7 f8 45 30 f1 fa 50 28 23 39 f1 ff-79 06 83 2d ea 72 15 85 3f 74 0d 22 01 8b c7 e8 ( I changed the bytes around)

Level 10
Report Inappropriate Content
Message 4 of 4

Re: A HIPS custom expert rule success story using Buffer Overflow, Caller_Module and target_bytes (undocumented stuff)

Wanted to let you all know that it seems target bytes are the pre-call and post-call values for the Windows API that caused the "buffer overflow" style alert:  (ctrl+f Structures)

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community