Author here.

The find-the-closest-finger algorithm was taken from another project we did, where we needed much more speed. We use a voronoi diagram with a distinct color for each point.. then we can get the pixel color at the position of the cursor and find the related pointer..

This is actually much faster than what you see happening on Pointer Pointer. We introduced a pause to force people to wait, thus making it a more rewarding experience when the image finally shows up. A touch of magic, you could say.