Generally the answers I've seen boil down to: our ML models is still pretty primitive. Models make their decisions in ways that aren't super generalizable, have decision frontiers that don't reflect the actual problem space well, etc. An example of such an analysis, see the "why is it hard to defend" section of https://blog.openai.com/adversarial-example-research/, or for a more academic approach which should be a good hook into the broader literature, see https://openreview.net/forum?id=rk6H0ZbRb