For silicon not to exist, the universe would have to be quite different. Maybe making life possible and in some planet a life form might eventually find out how to build computers with whatever chemistry they'd end up with. And the chemistry of such a universe would seem uncannily suited for such computers.

Point being, you can't delete an element from the universe and expect everything else to be the same. Silicon exists because of the physics in this universe. So do silicon based computers.