Skip to content

Comment on The "Soft Maximum" functionparent

Comments

As flipper points out, the problem is that the hard maximum is not differentiable; that is, the differential of max (x, y) is undefined when x == y. Most numerical approximation techniques require a smooth, differentiable function, so this is bad.

A related problem is that of bounding. Suppose I want to minimize the parameters in a set of nonlinear equations; I can numerically differentiate the equations to get the gradient and the Hamiltonian and then I can minimize that. But I may want to impose additional criteria, like x > 0. I could just say the function goes to some preposterously high number when x <= 0, but then we have this hard corner problem again. Instead if you use a continuous function like a logarithm to impose your bound, it affects the solution space minimally, makes for a solution that will not wander outside your bounds, and is differentiable.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.