I'm open to being convinced, but the study you've linked me is limited specifically to professionally written, proofread journalism from 8 publications over a span of a little over a single year.
It's not exactly a wide breadth of varied text for such an all-encompassing task.
The primary reason I am not convinced is because it (and several other tools) frequently classify my own writing as AI-generated. Especially my technical writeups.
Now, I could just assume I'm an outlier within that small false-positive rate. But I find it far more likely that the efficacy of these tools is much lower than advertised and they rely on confirmation bias.
Comments
I'm open to being convinced, but the study you've linked me is limited specifically to professionally written, proofread journalism from 8 publications over a span of a little over a single year.
It's not exactly a wide breadth of varied text for such an all-encompassing task.
https://www.pangram.com/blog/third-party-pangram-evals
You can also just try it yourself I guess really what convinced me was how it perfectly agrees with my own judgement.
The primary reason I am not convinced is because it (and several other tools) frequently classify my own writing as AI-generated. Especially my technical writeups.
Now, I could just assume I'm an outlier within that small false-positive rate. But I find it far more likely that the efficacy of these tools is much lower than advertised and they rely on confirmation bias.