The confusion is in a sense intentional as the whole point of Chomsky's program (Universal Grammar) is that natural languages essentially work like formal languages.

I don't think that this is actually true at all (and modern neuroscience, sociolinguistics, etc. disagree pretty heavily with Chomsky), but it is impressive how far you can get with modelling natural language grammar with formal methods.