It helps but you can't always assume unwanted logins are going to be brute force attacks. My biggest source of paranoia is a stolen/lost laptop with a saved SSH password/key. I prefer ACLs to port knocking. Depending how important a machine is there's no good reason to allow the entire Internet in. If it means someone has to drive to the office or do some SSH hoping that's a small price to pay.