I'll be the antiquated person here. Writing/speaking well has brought people to knowledge because the authors are uniquely positioned to use genius, humour, gentleness and generosity to bring an inquisitive but ignorant person into a new area of knowledge. When the value of that can be quantified, then we can compare AI "generation" of "efficiently written, useful knowledge" with what we had/have.
Same for fiction, visual art, raising children, caring for old people, and so on, and so on.
With big enough cohorts we can AB test by quantifying outcomes? Treatment group A has AI-generated texts and Treatment group B has the originals. We can have a questionnaire for how the group felt about the material etc, but we can also perhaps measure life outcomes over a bigger period e.g. performance at school or in the market place?
As I write this it feels naive and reminds me of a thousand ill-thought-out AB website tests etc. But still :D
There are plenty of very long-term studies going on where groups are identified and then followed over years or decades.
Another approach is today you might go looking for people with different levels of airborne lead exposure as a child, and then compare the average income today to see if their is correlation etc. In these kinds of studies the treatment groups weren't wittingly part of the experiment, but you can look backwards.
Either way, in 20 years we'll probably be able to identify schools where AI texts predominated earlier than other schools and, adjusting for other factors, try and tease out the actual impact of that choice.
I mean the thing is, ethically speaking, there is a strong supposition that the AI predominating instructional format will underperform the other formats, therefore you are essentially putting large groups of people, on purpose, in learning situations the theory is will underperform for the rest of their lives.
That seems like a bad thing.
I suppose that it will happen however, and the backwards looking thing will be what is done to determine the result.
Comments
I'll be the antiquated person here. Writing/speaking well has brought people to knowledge because the authors are uniquely positioned to use genius, humour, gentleness and generosity to bring an inquisitive but ignorant person into a new area of knowledge. When the value of that can be quantified, then we can compare AI "generation" of "efficiently written, useful knowledge" with what we had/have.
Same for fiction, visual art, raising children, caring for old people, and so on, and so on.
With big enough cohorts we can AB test by quantifying outcomes? Treatment group A has AI-generated texts and Treatment group B has the originals. We can have a questionnaire for how the group felt about the material etc, but we can also perhaps measure life outcomes over a bigger period e.g. performance at school or in the market place?
As I write this it feels naive and reminds me of a thousand ill-thought-out AB website tests etc. But still :D
exactly how long are you planning on running these tests for - sounds like minimum 20 years?
Yeap :)
There are plenty of very long-term studies going on where groups are identified and then followed over years or decades.
Another approach is today you might go looking for people with different levels of airborne lead exposure as a child, and then compare the average income today to see if their is correlation etc. In these kinds of studies the treatment groups weren't wittingly part of the experiment, but you can look backwards.
Either way, in 20 years we'll probably be able to identify schools where AI texts predominated earlier than other schools and, adjusting for other factors, try and tease out the actual impact of that choice.
I mean the thing is, ethically speaking, there is a strong supposition that the AI predominating instructional format will underperform the other formats, therefore you are essentially putting large groups of people, on purpose, in learning situations the theory is will underperform for the rest of their lives.
That seems like a bad thing.
I suppose that it will happen however, and the backwards looking thing will be what is done to determine the result.