From the data for pre-GPT-era pages, it looks like a false positive rate of about 1.2%. So when it says that 35% of current pages (pages published, say, this month) are detected as AI-written, 33.8% of them probably actually are.
No. Of pages written before GPT, so 100% of pages were not written by GPT, the detector said that 1.2% of them were written by GPT. So 100% innocent pages, but 1.2% marked as guilty. But of the 1.2% marked, 100% of them were false positives.
But my calculation was wrong too. What I should have said is, 65% were non-GPT pages, but 1.2% of those were getting flagged, or 0.78%. So 34.3% were actually written by GPT, to first order.
Comments
From the data for pre-GPT-era pages, it looks like a false positive rate of about 1.2%. So when it says that 35% of current pages (pages published, say, this month) are detected as AI-written, 33.8% of them probably actually are.
1.2% of positives were false, right? Not 1.2% of the entire dataset.
So it would be 1.2% of that 35% positive subset. So 34.65%.
No. Of pages written before GPT, so 100% of pages were not written by GPT, the detector said that 1.2% of them were written by GPT. So 100% innocent pages, but 1.2% marked as guilty. But of the 1.2% marked, 100% of them were false positives.
But my calculation was wrong too. What I should have said is, 65% were non-GPT pages, but 1.2% of those were getting flagged, or 0.78%. So 34.3% were actually written by GPT, to first order.