Though APL may strike some as a strange language of choice for deep learning

It sure did to me, even as someone who has written (a trivial amount of) J. But the argument that follows is more than convincing.