Comment on Show HN: Crawl a modern website to a zip, serve the website from the zipComments−unlogOP2yI'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have been experimenting with crawling myself for uploading to hosts without having to do SSR. This is the result−mikeqq20242yfor a long crawling task, if exited/broken for any reason, does it save and restore at the next run?−eXpl0it3r2yThe README says:Can resume if process exit, save checkpoint every 250 urls−mikeqq20242ynice, better make it as a commandline option with default value. 250 is too many for large files and slow connection.
Comments
I'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have been experimenting with crawling myself for uploading to hosts without having to do SSR. This is the result
for a long crawling task, if exited/broken for any reason, does it save and restore at the next run?
The README says:
nice, better make it as a commandline option with default value. 250 is too many for large files and slow connection.