Besides the outstanding issues of ESM not being able to parse the large TAXII files it receives, I would suggest to try to enter "system.Default" for the collection value.
When I was having issues connecting into my TAXII service, I ran tcpdump on the receiver to confirm that everything with the packet structure is sound. You should be able to determine exactly what's causing your TLS issue by going about this troubleshooting method.
Here at my work place I implemented a different way about using TAXII with ESM. I would suggest you use Soltra (soltra.com) to facilitate the storage of threat intel you receive from NH-ISAC then plug ESM into your on-site TAXII instance.
We are seeing the same error with an on-prem TAXII server . The connection works fine with SSL disabled (http) on the TAXII server, but fails with the “Error, Handshake Alert: Unrecognized_Name” error when enabled (https). It looks like a Java 1.7.0 feature;)
I have opened a ticket with McAfee support and suggest you do the same so we can get some traction on this.