By that logic, you are never allowed to notice the unbalanced application of a rule or principle.

Slightly off-topic, most "logical fallacies" actually have very important heuristic uses. Just to get the ball rolling, "ad hominem" becomes extremely useful in a world in which we have limited access to full information, and have to constantly evaluate the reliability of outside parties.