building from source is not going to fix the problem. Nobody reads the source code of projects they download and compile themselves

However commits tend to be much easier to trace at a later date than arbitrary binaries so attackers will be less inclined to go that route. Once committed it's there forever unless you can somehow get everyone to censor it from their own copies for an unrelated reason. Consider that the xz compromise involved downloading the payload later.

My policy is to either obtain binaries from a major distro or to build from a clean commit in a network isolated environment. If I can't go one of those routes it's almost always a hard pass for me.