Comment on SiteOne Crawler – big tool with no usersparentComments−internetter1yVery small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?−mistaken1yIt seems to contain the crawler: https://github.com/janreges/siteone-crawler/blob/main/src/Cr...−janreges1ySiteOne Crawler offers 2 open-source tools: 1) command-line version (which is the core of the crawler itself): https://github.com/janreges/siteone-crawler 2) a graphical application for Win/macOS/Linux, which is just a wrapper over the command-line version and displays the output from it through Xterm: https://github.com/janreges/siteone-crawler-gui
Comments
Very small amount of sourcecode: Under 20k by my count. Are they outsourcing the actual crawl to a different tool?
It seems to contain the crawler: https://github.com/janreges/siteone-crawler/blob/main/src/Cr...
SiteOne Crawler offers 2 open-source tools: 1) command-line version (which is the core of the crawler itself): https://github.com/janreges/siteone-crawler 2) a graphical application for Win/macOS/Linux, which is just a wrapper over the command-line version and displays the output from it through Xterm: https://github.com/janreges/siteone-crawler-gui