Google is, to a first approximation, the web's only search engine. As such I think there's an argument to be made that declaring your bot's User-Agent as Googlebot is a technical necessity, similar to how every browser declares itself as Mozilla.
They aren't alleging that it is illegal. They are alleging that it violates terms that Bright Data agreed to when Bright Data signed up for Twitter.
How do website terms of service work in Switzerland, especially in the case where the person using the website has actually gone through the account creation process there?
Much of the crap put into terms of service is null and void. It can't restrict your legal rights. As an example, they can't force you into arbitration if you want to sue them.
How the internet survives when basic rights conflict is the active question. 202x is the decade ever nation on the planet woke up & decided to start regulating what happens across the internet.
And most of these rules don't respect anyone else's. Your people in your country do whatever you say whenever they travel across the internet afar. But those people running their servers also have their own laws that apply to them. The potential for conflict here is vast.
interesting point. in which countries that would be 100% legal in general? BrightData is set up in Israel, right? Is that the case there? But I wonder if having a US entity (NY) they are bound to US jurisdiction for this. Also, what does that mean for all tooling that scrapes/d Twitter?
Comments
Good luck with that in Switzerland where this is 100% legal.
Or in Germany, see §44b UrhG.
[1] https://www.gesetze-im-internet.de/urhg/__44b.html
It seems to me that Twitter's robots.txt only allows Googlebot:
https://twitter.com/robots.txt
Therefore, this disallows other bots in "maschinenlesbarer Form" and the scraping is illegal.
Google is, to a first approximation, the web's only search engine. As such I think there's an argument to be made that declaring your bot's User-Agent as Googlebot is a technical necessity, similar to how every browser declares itself as Mozilla.
This is fascinating. Seems like at least Bing openly ignores robots.txt?
They aren't alleging that it is illegal. They are alleging that it violates terms that Bright Data agreed to when Bright Data signed up for Twitter.
How do website terms of service work in Switzerland, especially in the case where the person using the website has actually gone through the account creation process there?
If it's like in Europe, if a contract is illegal or restrict rights disproportionately, it's unenforceable.
Much of the crap put into terms of service is null and void. It can't restrict your legal rights. As an example, they can't force you into arbitration if you want to sue them.
How the internet survives when basic rights conflict is the active question. 202x is the decade ever nation on the planet woke up & decided to start regulating what happens across the internet.
And most of these rules don't respect anyone else's. Your people in your country do whatever you say whenever they travel across the internet afar. But those people running their servers also have their own laws that apply to them. The potential for conflict here is vast.
interesting point. in which countries that would be 100% legal in general? BrightData is set up in Israel, right? Is that the case there? But I wonder if having a US entity (NY) they are bound to US jurisdiction for this. Also, what does that mean for all tooling that scrapes/d Twitter?