I think GIFs only have one rectangle that differs per frame so a "dumb" encoder would be replacing nearly the whole frame each time, since your changes are small but distributed all over the frame.

The smarter encoders would encode most of that rectangle as transparent and let the previous frames show through.