Based on my equally limited sample size (based largely on looking around while working on my Masters in math) I disagree with you findings. Lots of people in the math departments where physically active, certainly more so than where musical (although a there was a fair bit of overlap).