It is important to understand that the user-agent isnt always present.
For example in transparent setups, it will not be present for HTTPS requests (because that information will be encrypted). I suspect your issue only exists for HTTPS, not for HTTP.
Check out my auth guide for "ideal conditions":
Perhaps you can use this to authenticate users for a longer interval, but re-authenticate users more often if they support it.