I had the same problem launching a java application that uses JNLP to launch the program. I kept getting an unable to create virtual machine error. What I ended up having to do was to exclude javaw.exe from the Application Protection Rules on my IPS Rules policy. Which is not good because now it leaves javaw.exe vulnerable. But at least the application will start successfully.
This was the problem I was having: