4 Replies Latest reply on Feb 16, 2015 9:59 PM by fuzzychaos

    NTVDM application error after Solidifier Command Line install


      The error that I see is similar to this: https://community.mcafee.com/thread/10342


      Copying the cmd.exe from a working system without Solidcore installed temporarily fixes the problem but this is not a solution.  This problem is affecting dozens of systems running Windows XP SP2/SP3.  These are embedded robots on a manufacturing line, so changing the software or OS is highly unlikely.


      The NTVDM error is:

      The NTVDM CPU has encountered an illegal instruction.

      CS:0000 IP:0077 OP:f0 37 05 0c 02 Choose 'Close' to terminate the application.

      Buttons --> 'Close'  'Ignore'.  Ignore doesn't help and Close forces the application to stop.


      Uninstalling Solidcore doesn't fix the problem, the PC has to be re-imaged and McAfee reinstalled without the Solidcore to keep it running.


      The application that is crashing is a 16-bit app (100s of apps) and uses the vector 66h 'Get or set code page' from the DOS INT 21h services in order to do some inter-process communication through a named pipe between the NTVDM DOS app and Windows applications which are series of C/C++/VB/Java programs all of which are running on the target PC.  There are 2 TSRs in the DOS NTVDM that take care of the communication with any other DOS app requiring it.


      The application that is crashing belongs to my company (the one that I work for) and our customer is running McAfee with Solidcore on 1000s of PCs.  They've asked me to investigate the root cause and provide any actions for them to configure on their end to fix it along with my confirmation of the fix.  I'm not familiar with the Solidcore software so please point me to any tips/workarounds that I can provide to the customer's IT dept to help with the configuration and investigation.


      Is there some filter setting to include/exclude/whitelist to enable  the system to allow execution and not crash the software?  Any known reason why the  Solidcore would cause the applications to start crashing like blocking named pipes for example?


      Previously KB2707511 security update from Microsoft caused a similar problem but Microsoft subsequently fixed it over a year ago for everyone.


      Thanks in advance for any assistance.



        • 1. Re: NTVDM application error after Solidifier Command Line install

          Hi fuzzychaos,

          If you do not really know Solidcore and there are 1000s of systems using it in your environment, I would hope that someone in your organization supports it??? To answer some of your questions, yes, App Control is likely blocking stuff and it is generating events that need to be reviewed. Upon review, configuration changes would then need to be made in the policies being applied to those systems. Solidcore is a dynamic whitelisting application in that it takes inventory of the system in its current state and then locks it down saying only this list of apps and files can be modified.... but there are configuration changes that can be made after the fact (hence the term: "dynamic") that will allow new apps to run or existing apps to udpate etc... You really need to understand Solidcore though in order to do these things. You will either need to mange them locally (man I hope not with that many systems) or you need to manage it via ePO.

          Does that help at all?




          • 2. Re: NTVDM application error after Solidifier Command Line install

            Yes, they have a team of people supporting the systems and I'm sure they would be using ePO.  No apps are updated but it would seem that the whitelisting is not allowing inter process communication somehow originating from the NTVDM.  The whitelisting inventory of files is probably fine since the files are not changing aside from a few logs.  Where are the Solidcore logs generated/stored, locally or on a central server?  What command would be run so that we can review the logs to try to see what it's generating?  Once we find the offending option it can be disabled as necessary I hope.

            • 3. Re: NTVDM application error after Solidifier Command Line install

              That's good to hear. You should be able to ask them and their ePO admin should be able to view the events in their ePO Console. There are logs located on the local system, the most useful being the Windows Application events... especially for blocked events, but here are the Solidcore generated ones:


              • Solidcore version previous to 6.0: <installation-dir>\Program Files\McAfee\Solidcore\Logs
              • Windows 2000, 2003, XP: <installation-dir>\Documents and Settings\All Users\Application Data\McAfee\Solidcore\Logs
              • Windows 7 (and later): <installation-dir>\Program data\McAfee\Solidcore\Logs
              • 4. Re: NTVDM application error after Solidifier Command Line install

                Dozens of re-imaged machines later... I'm marking the logs as correct because the logs helped to narrow down the problem after many months and the problem seems to be resolved after adding the blocked executables to the 'exceptions list'.  I haven't received further feedback so I'm guessing that this is not an issue for them any more.