Legal issues aside, I'd think of it this way:

If you walked in to work tomorrow and got fired on the spot ("Give me your laptop and leave the building"), would you be comfortable just handing over your machine, then turning around and walking away? If not, that's a problem. It's a problem for you, and a problem for your employer.

Best to keep things as separate as they can be.