If you’re working with a touch display, perhaps a hold-and-drag-to-scroll feature could work? Have no indicator when pressed and only refresh the screen when the user lifts their finger.

I like this idea, definitely going to test it out.