If people feel that benefits aren't going strictly to the intended recipients the sense of fairness that underlies the system is eroded.

And in an ideal world, you then say "that will literally cost you more money than not doing it", and the argument ends.

In practice, gatekeeping on government programs is primarily pushed for by people who don't think the programs should exist at all. The Venn diagram of the set of people who will withdraw their support for programs without such gatekeeping, and the set of people who mostly don't support such programs in the first place, is close to a circle.