> but this story seems to illustrate a "waterfall" process, executed relatively successfully 50 years ago. Why then, are we so bad at it now?

We have much smaller budgets, and our budget sizes are not classified.