I'm trying to launch a JNLP file and keep getting an error
If I disable HIPS by unchecking the "Enable Host IPS" checkbox, the program launches successfully.
But if I keep "Enable Host IPS" checked, and instead I uncheck all the HIPS Engines (Help -> Troubleshooting -> Functionality), I still get the unable to create Java Virtual Machine error.
There are no HIPS events being triggered and there is nothing helpful that I can find indicating the problem in the HIPS log files even in Debug mode.
Any ideas on where else I can look to troubleshoot?
Open a ticket with McAfee. They will probably have you set all your McAfee default policies for IPS, but also with your Client UI policy set to debug. Reproduce issue, note the exact time, and run a MER.
"They will probably have you set all your McAfee default policies for IPS"
Thanks, that helped resolve the problem (sort of). So I tried setting the IPS Rules policy to only the McAfee default and it still didn't run. So I did the opposite. I applied every policy EXCEPT for the McAfee Default policy and it worked!
Then I started looking for differences between the McAfee Default IPS signature settings and the Effective Policy when McAfee default wasn't applied. They were both the same. But I was only checking the signatures containing the word "java".
Then I checked the Application Protection Rules and searched for "java". The difference when applying the McAfee default and not applying it was that javaw.exe was not on the application protection list (only java.exe and javaws.exe were).
So for now, I'm going to have to assume it has something to do with IPS blocking javaw.exe. I even disabled all the signatures containing the word "java" but it still didn't run with the McAfee Default policy applied so it seems like protecting javaw.exe is causing the block.
Your IPS Rules and Trusted application policies should be running with McAfee Default policy, along with any custom policy, per McAfee Best Practices. These are multi-slot policies.
PD22894 - Host Intrusion Prevention 8.0 for ePO 4.5 ProductGuide
Assigning multiple instances of the policy
Assigningone or more instances of the policy to a group or system in the ePolicyOrchestrator
SystemTree provides for single policy multi-purpose protection.
TheIPS Rules policy and the Trusted Applications policy are multiple-instancepolicies that can
havemore than one instance assigned. A multiple-instance policy can be useful foran IIS
Server,for example, where you might apply a general default policy, a server policy,and an
IISpolicy, the latter two configured to specifically target systems running as IISservers. When
assigningmultiple instances, you are assigning a union of all the elements in eachinstance of
NOTE: The McAfee Default policy for both IPS Rules andTrusted Applications are updated when
content is update.McAfee recommends that these two policies always be applied to make sure
protectionas up to date as possible.
Check for Signature violations for Javaw.exe, once HIPS is injecting into that process.
KB67056 - Third-party application stops working or isimpaired after McAfee Host Intrusion Prevention is installed or content isupdated
Thanks KB67056 has some good info. I did most of what that article said to do. The main problem was that there was no info in the log files even in debug mode that I could find useful information from.
You make a good point in that we shouldn't be disabling the McAfee Default policy since that is the one that gets the updated content.
So first I'll try enabling the McAfee Default policy and add javaw.exe to the Trusted Application policy.
If that doesn't work, I will try to do like KB67056 says:
Disable Signature 432Message was edited by: fuzziest
So this is what I had to do to get it running with the McAfee Default Policy applied.
I copied the javaw.exe entry from the McAfee Default policy application protection list tab to our custom policy for the group.
Then I modified the javaw.exe entry on the custom policy and set Inclusion Status to "Excluded".
Now it works.
It makes sense to do it this way, but it looks kind of funny when I view the "Effective Policy" for the group (shown in screenshot above).
Two of the same rule, but one shows Included and one shows Excluded.
I would expect it to only show the "effective" rule taking effect, which in this case would be the "excluded" one.
Otherwise, how would I know which rule is being applied?
Anyway, it works so I'm happy.