You will not be able to do a configuration update to the sensor when using 18.104.22.168 and a 5.x sensor as that is not supported.
The incompatible signature set is because it is a version not applicable to the sensor version. What signature set is loaded on the manager?
The easiest solution is likely to do a 'loadimage' on the cli over tftp as that should work without any restriction.
So if the manager is 8.0.59 and the sensor is 6.1.1.x I cannot push an update from the Manager. I can get the sensor up to the 8.0.x version but the only way to do this is through tftp directly on the sensor correct? Or do I have to stay in the 6.1.1.x 'family' of updates for both the sensor and manager?
8.0 can manage 7.1, 7.5 and 8.0 sensors. If your sensor is at 6.1, then yes, you will need to use loadimage on the CLI to get it to one of the supported versions before you will be able to do an update through an 8.0 manager.
This is covered in the release notes: http://kc.mcafee.com/corporate/index?page=content&id=PD24782