Comment on Show HN: I built a dataset of 25k+ Gumroad products to spot market opportunitiesComments−mateuszbuda2yHow did you collect this data? Did you just use webscraping? What tools did you use for this?−topofthefortsOP2yYes just webscraping. Just used Laravel, cause that's what I'm most familiar with. The site is Laravel + AlpineJS (with Vite)−AbstractAirways2yI tried to scrape them and found they had some pretty robust anti-scraping implemented. Did you have to do anything to overcome it?−mateuszbuda2yExactly, did you build your own solution for this, maybe based on open source projects, or do you rely on external service for scraping? I'm curious too since I'm developing a web scraping API product (https://scrapingfish.com).−KomoD2yWhat measures?
Comments
How did you collect this data? Did you just use webscraping? What tools did you use for this?
Yes just webscraping. Just used Laravel, cause that's what I'm most familiar with. The site is Laravel + AlpineJS (with Vite)
I tried to scrape them and found they had some pretty robust anti-scraping implemented. Did you have to do anything to overcome it?
Exactly, did you build your own solution for this, maybe based on open source projects, or do you rely on external service for scraping? I'm curious too since I'm developing a web scraping API product (https://scrapingfish.com).
What measures?