Showing results for 
Show  only  | Search instead for 
Did you mean: 

Buffer Overflow - Tracing the root of the problem

I'm sure everyone knows by now that Patch 4 extended BO into DEP detection, and people are getting BO's when they didn't before.

I'm happy to accept this as working as expected - any program that has a BO could be exploited, so we NEED to know about them. You may have a different opinion, but I wanted to put that aside for this thread.

If a program shows a BO, we are told to ensure we have the latest version, all patched - but what if we already do?

For example, Acrobat Reader has this problem on a users PC. This is the latest version. Obviously there is something else affecting this (the problem doesn't happen on the same machine as another user).

What would be the best way for attempting to find out what DLL/add-in/hook is causing the BO? Any tools? Simply Process Explorer?

1 Reply
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 2

Re: Buffer Overflow - Tracing the root of the problem

I was curious to see if anyone else would respond

As for the "best way" to discover the offending DLL, I don't really have the answer.

Process Explorer is certainly a good start. It may not be enough though. It'll certainly show if any 3rd party DLLs are present within the process that you weren't expecting, or weren't aware of, and perhaps that's sufficient to provide you a lead - as in, check if the process still triggers violations when those DLLs aren't being loaded (removing the app, or renaming the DLLs, or using VSE's Access Protection to stop the DLLs from being Read... muhahaha)

In Support's experience, this method allowed us to easily spot some culprits that were hooking the likes of Explorer.exe, or IExplore.exe etc.

Another approach which is doable outside of Intel Security, is to disable VSE's BOP but enforce DEP for that process. You'll still get a DEP violation from that process, but you can also now get a process dump showing why DEP made that conviction (as DEP halts execution of that process). Looking at the dump in your favorite debugger (e.g. WindBg) may reveal the offending stack, and show you what modules were making API calls prior to DEP intervening.

I've used this method before to discover a DLL that was knowingly DEP compatible was checking if DEP was enabled by violating DEP! Hilarious.

Otherwise, there is 1 more method that has worked for me which could only be done by Intel Security -

We have a debug tracing tool, called ETLTrace.exe - I'm sure some of you have had to use it - which captures Event Tracing for Windows (ETW) data of our Providers. The trace can tell us which DLLs were recently "touched" prior to the violation report - unfortunately it can't tell us which DLL caused the violation because we don't get that data (it's all handled by DEP) we only get the notification from DEP that a violation occurred.

Still, by reviewing the recent DLLs that were active, we can form a list of suspects to do further testing/verification with and prove the theory... in a VM ideally, because the method for proving is brutish, getting rid of the DLL and see if violations still occur.

William W. Warren | S.I.R.R. | Customer Success Group | McAfee
You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community