the maybe easy way will be to use the migration tool like mentioned in the install document. But in that case you may have to make changes to your production environment (eg remove old extensions) you don't want to do.
So a nother way would be to move the DB to a new server AND ePO 4.6.6 to a new server and then doing the Upgrade to 5.1
This is not possible. You cannot install ePO 4.6 on a Windows Server 2012 R2 (not supported). The only way to do this is a transfer of systems and policies, etc.
You have to register the new ePO 5.1 server on the old ePO 4.6 server. You can use different SQL databases, whatever you like. Keep in mind, you have to migrate the security keys (export/import) both 1024 and 2048 keys for this to work. After you registered the new server to the old ePO 4.6 server, you can begin transfering systems. A new option is there for you in the system tree. It takes a while for all systems/clients to be moved. After this I exported the policies and assignments and the rest.
It is not that hard, if you put some work in it. It just takes some time.