Hm, it’s still a tricky question. Do web admins have a right to block agentic AI? Of course they do. They have a right to block whoever they want. Distinguishing between scraper and agent is important, yes, but agents can still produce a lot more traffic than the average human, and it’s not like the agents are optimizing their access patterns.
i agree. i think the point i agree with perplexity on is that CF is a central authority that is claiming to be the gatekeeper while (allegedly) being disingenuous and incapable.
i also tend to agree with the concept that scraping != consuming on behalf of a user. they explicitly point out that they do not store the data for training, which would fall under scraping by proxy.
Comments
Hm, it’s still a tricky question. Do web admins have a right to block agentic AI? Of course they do. They have a right to block whoever they want. Distinguishing between scraper and agent is important, yes, but agents can still produce a lot more traffic than the average human, and it’s not like the agents are optimizing their access patterns.
i agree. i think the point i agree with perplexity on is that CF is a central authority that is claiming to be the gatekeeper while (allegedly) being disingenuous and incapable.
i also tend to agree with the concept that scraping != consuming on behalf of a user. they explicitly point out that they do not store the data for training, which would fall under scraping by proxy.