When I read the headline I imagined they had implemented back propagation in a physical system

They touch on that by observing you could train a second physical neural network to compute the gradients for the first. So it could all be physical.

Improvements to PAT could extend the utility of PNNs. For example, PAT’s backward pass could be replaced by a neural network that directly estimates parameter updates for the physical system. Implementing this ‘teacher’ neural network with a PNN would allow subsequent training to be performed without digital assistance.

So you need to use in silico training a at first, but can get rid of it in deployment.