True. To keep the explanation simple I glossed over some details. The problem you describe exists even in the 1D case, e.g. the function f(x) = 0. However if a convex function has multiple minima then the minima will be connected and at the same height, so the algorithms still apply.