IPS Rules policy (should be) are compatible with ePO 5.1, but make sure you are running the same HIPS extension between ePO servers. This is required.
KB76900 - Mismatched Extension versions on ePO servers may cause policy corruption and other undesired behavior
I will take a look at this. I installed a new epo server but imported existing policies.
Funny thing is, on another server I only added the HIPS extensions. On this server the HIPS IPS license also failed.
There is no newer version of the HostIPSLicense file.
All Host IPS 8.0 License Extension for ePO 4.0