In terms of scraping, there's already a huge project that collects business data in an OSM-compatible format: http://alltheplaces.xyz/
The main trouble is licensing and change tracking. Most of the scraped data is protected by copyright or database rights so it can't be imported.
And even if the licensing is solved, you have the problem of matching scraped data to OSM data and what to do when changes disagree. For example, a store might be scraped as a point in the middle of a shopping mall, but then an OSM editor would come by and move it to the correct section of the mall - the next import round shouldn't undo that. Or maybe a store changes opening times but forgets to update their website - an editor can fix that, but the next import would break it again.
I have a sort of "grey area" idea for this, but I haven't had the time to try it. Basically, I would track changes in AllThePlaces and create "change reports" such as "store X changed open times from AAA to BBB". Then, I'd make a UI that would show you the changed website alongside an OSM editor and a convenient "copy change" button.
This way, a human is still the one looking at the website and entering info into OSM, which is essentially the same as in-person surveying. The copy button is "just a convenience".
Still, I think this is too messy from a legal standpoint and the OSM editors wouldn't allow it out of caution...
Thanks for the great answer. Based on what I'm reading scraping a business website is probably fine in general but using data from Google Maps/Yelp/etc is generally not gonna fly.
What I think could work is if everydoor allows you to create notes with photos (https://github.com/Zverik/every_door/issues/184) then it would be pretty easy to later go back and drop those photos into an AI tool and extra websites and try to create some tags for review. Could also work with Streetcomplete but there it's not easy to see if a POI already exists.
In any case, I might experiment with this idea further.
Some very basic testing shows me that Claude 3.5 Sonnet is pretty great at taking a photo of a menu turning it into decent tags.
So if I could run around taking photos of menus and the outside of businesses then quickly turn them into tags later that would be a nice workflow for me (and hopefully others).
Comments
In terms of scraping, there's already a huge project that collects business data in an OSM-compatible format: http://alltheplaces.xyz/
The main trouble is licensing and change tracking. Most of the scraped data is protected by copyright or database rights so it can't be imported.
And even if the licensing is solved, you have the problem of matching scraped data to OSM data and what to do when changes disagree. For example, a store might be scraped as a point in the middle of a shopping mall, but then an OSM editor would come by and move it to the correct section of the mall - the next import round shouldn't undo that. Or maybe a store changes opening times but forgets to update their website - an editor can fix that, but the next import would break it again.
I have a sort of "grey area" idea for this, but I haven't had the time to try it. Basically, I would track changes in AllThePlaces and create "change reports" such as "store X changed open times from AAA to BBB". Then, I'd make a UI that would show you the changed website alongside an OSM editor and a convenient "copy change" button.
This way, a human is still the one looking at the website and entering info into OSM, which is essentially the same as in-person surveying. The copy button is "just a convenience".
Still, I think this is too messy from a legal standpoint and the OSM editors wouldn't allow it out of caution...
Thanks for the great answer. Based on what I'm reading scraping a business website is probably fine in general but using data from Google Maps/Yelp/etc is generally not gonna fly.
What I think could work is if everydoor allows you to create notes with photos (https://github.com/Zverik/every_door/issues/184) then it would be pretty easy to later go back and drop those photos into an AI tool and extra websites and try to create some tags for review. Could also work with Streetcomplete but there it's not easy to see if a POI already exists.
In any case, I might experiment with this idea further. Some very basic testing shows me that Claude 3.5 Sonnet is pretty great at taking a photo of a menu turning it into decent tags.
So if I could run around taking photos of menus and the outside of businesses then quickly turn them into tags later that would be a nice workflow for me (and hopefully others).