The inverse happens too. Racists and sexists define everyone who they don't agree with as Social Justice Warriors. Of course, neither end of the spectrum think that the corresponding label applies to them.