Great explanation of the intuitive understanding of the path kernel which seems to be the main takeaway from this paper.
One minor technical correction, the proof relief on the continuous model gradient flow not SGD. So it’s proven for GD and likely true for SGD and your intuitive explanation likely still holds, but it’s not obvious.
Comments
Great explanation of the intuitive understanding of the path kernel which seems to be the main takeaway from this paper.
One minor technical correction, the proof relief on the continuous model gradient flow not SGD. So it’s proven for GD and likely true for SGD and your intuitive explanation likely still holds, but it’s not obvious.
You're absolutely right. I substituted SGD for GD without giving it any thought because everyone uses SGD!
They sketch the argument for SGD, but they don't know if it actually holds (see Remark 5 in the paper).