I think if you read the release notes for each version you'll see a different picture:

Vite 2 was a big architectural change, but Vite 3 and 4 have been driven by external changes -- node EOL, major rollup release, browser releases.

The amount of actual code churn is fairly low and dropping.