Thanks, technically it scrapes every 10 minutes the first 3 pages of HN and the first 3 /newest, fires up a Chrome instance with puppeteer, visits each site with a spoofed GoogleBot UserAgent to avoid paywalls, scrapes `document.body.textContent`, sends it to openrouter/free and asks to summarize. We also collect the opengraph images to show in that step. One difficulty was getting the responses to not be garbage enough but with some prompt tweaks that fixed. It runs on a small VPS with bun. Biggest challenge was setting up the deployment pipeline actually because I hadn't done it before. But now I can `git push` and it gets updated and restarts fine :)
Comments
Pretty nice, Can you share some tech details and challenges
Thanks, technically it scrapes every 10 minutes the first 3 pages of HN and the first 3 /newest, fires up a Chrome instance with puppeteer, visits each site with a spoofed GoogleBot UserAgent to avoid paywalls, scrapes `document.body.textContent`, sends it to openrouter/free and asks to summarize. We also collect the opengraph images to show in that step. One difficulty was getting the responses to not be garbage enough but with some prompt tweaks that fixed. It runs on a small VPS with bun. Biggest challenge was setting up the deployment pipeline actually because I hadn't done it before. But now I can `git push` and it gets updated and restarts fine :)