The difference is that in SIA you can't "try it on a non-differentiable function" because in SIA, such a thing doesn't exist.

It's not like the C case where out of bounds access is assumed to not exist, but they can happen anyway and if they do, results are catastrophic. In SIA an analogous catastrophe doesn't happen - it's as if it had a type checker that will reject nonsensical results.

Such a "quadratic bound" approach (in brief f'(x) exists if there exist a constant C and a neighborhood of zero where for all h in the neighborhood |f(x + h) - f(x) - f'(x)h| ≤ Ch²) could actually be adopted over the typical limit of (f(x + h) - f(x))/h without abandoning classical logic, but considering the difficulty of proving the bound for many functions of interest, it might not make much of a difference over just assuming differentiability either way.

Now this I don't understand. I think that what changes in SIA isn't just the definition of a derivative, but being able to use infinitesimals to simplify the calculations