I am afraid you are the one who is not showing signs of having thought about this deeply. What is the ratio of the number of application programs, libraries, and services to the number of VMs and runtimes? Thousands, tens of thousands, millions? Depends on how you count, but it's huge. Reducing the attack surface like this is a big win.

And it is indeed a bad idea to install a browser on a critical server, and to load untrusted sites in it. You can mitigate the problem by not doing that. You can't stop the server from dealing with user data, though, since for many servers, that's what they are for. (If you are not going to deal with untrusted data, it is preferable to disable untrusted connections at as low a level as you can manage).