Errrrr... how is the first paper related to A* and path finding?

It's about "planning" (ie. searching an action space) which does not generally have a heuristic, so it needs to do "exploration" (which is exactly the opposite of what you want for optimal path finding) to counteract getting stuck in local optima due to greedy search...