Thinking about evaluation at the beginning, not at the end of a project, means that you don't fly blind during most of the project's execution duration.
Yes, I've been working in the machine learning space for a while and I agree that it's an approach that should've been followed years ago. But I also think that now the problems of not following it are bigger, because now we have "AI as a service" so everyone can have AI in their systems.
Comments
What is called "Evaluation-Driven Development" (EDD), I called "evaluation first" approach: https://arxiv.org/abs/2201.07725
Thinking about evaluation at the beginning, not at the end of a project, means that you don't fly blind during most of the project's execution duration.
Yes, I've been working in the machine learning space for a while and I agree that it's an approach that should've been followed years ago. But I also think that now the problems of not following it are bigger, because now we have "AI as a service" so everyone can have AI in their systems.
I was there around 2012.