The right way to do this IMO is big version upgrades. Like V1 of protocol uses a specific hash, message format, etc., V2 might use another, etc. It's when you allow arbitrary combinations of pieces that complexity becomes absurd.

A lot of this happened due to the ITAR export crap where you needed to offer pluggable keylengths and algorithms for compliance, and then people had the insane idea of supporting arbitrary algorithms in every protocol.