This is useful analytically, but one tradeoff here is that if this is in an inner loop, the exp/sum/log process will be a lot more expensive than a simple max.
For proofs and stuff it can be invaluable to have a differentiable objective function, but in actual computation it's not always necessary.
Depending on your application, a good compromise might be to use soft max if the numbers are nearby and a hard max if they are very distant (though you'd need to find a suitable definition of distant and nearby).
Comments
This is useful analytically, but one tradeoff here is that if this is in an inner loop, the exp/sum/log process will be a lot more expensive than a simple max.
For proofs and stuff it can be invaluable to have a differentiable objective function, but in actual computation it's not always necessary.
Depending on your application, a good compromise might be to use soft max if the numbers are nearby and a hard max if they are very distant (though you'd need to find a suitable definition of distant and nearby).