There's an easy way, actually: Don't do publically visible things in the first place. Certainly, this limits your options in life, but if you simply can't tolerate other people speculating about you, reasonably or otherwise, you don't give yourself much choice.