Absolutely apply. We have a bunch of people that don't really care about physical stuff at all, and would rather just think about the problems.

That said, the classes aren't going to all be about math and only math, a lot of classes have physical projects. But the projects are tightly integrated with theory. I learned multivariable calculus by modeling a boat with some curves, and then building the boat out of foam to see if the calculated angle at which it tips was the same as reality.

If you kid can get excited about project-based learning, Olin is an awesome option.

I'd be happy to talk to you more if you have more questions.