Forgive me if this is not applicable. I skipped ePO 4. Went straight from 3.6 to 4.5.
That said, if you are not performing AD sync actions, I would create new system tree folders as needed and assign the client tasks that deploy the new stuff to them and set them to run immediately.
Then I would move the systems that I wanted to upgrade the Agent on to the "Agent" install folder I just created and disable system tree sorting on them.
Then I would wake them up. At that point they should check-in and pull the new tasks and execute them.
When finished with them, I would re-enable system tree sorting and then wake them up again to send them back to where they were prior.
If you are performing AD syncs and pulling in the AD structure, your new folders will get wiped-out. Turn off the AD sync task until you are finished.
Hope that helps.