It could take 30 seconds and up till the browser switches the connection. We tested it last year and decided to have the VIPs every time running. So, we have four VIPs at one LB-Cluster (4 nodes). With this config three LBs could be "out of order", but all VIPs are online.
Thanks freddykr. We are going to do some testing as well. It'll be interesting to see the results.