An interview process is successful if you hire candidates which turn out to be a good fit both ways.

An interview process is fair if you don't exclude any candidates that would have met the criteria if not for the lousy interview process.

While you seem to be enjoying the former (for now), you are intentionally or accidentally narrowing the pool, and mistreating a number of your candidates.

If intentional, the word will get out, even if it takes a while. If accidental, I hope you work to improve the process.