Can someone give me examples of where you use agents in your browser? I’ve heard executive leaders tout that “people use agents to buy things for them” but I haven’t actually seen that.
I’ve heard executive leaders tout that “people use agents to buy things for them” but I haven’t actually seen that.
My wife really dislikes building up the shopping cart for our weekly grocery delivery, so I built an agent... thing with earendil's npm libs. It takes the menu my wife has decided on, confers with her about the ingredients (if it hasn't seen a recipe before), and then uses Chrome's devtools protocol to head to Walmart and add everything to the shopping cart.
It works fairly well and uses the local models I have running on my Mac Studio.
Yeah, they're excellent. I use a small Qwen3-4B-Instruct as the "actor" agent which goes out to the website, searches for a batch of items, decides which best matches the criteria, and puts them in the cart. Then I have Qwen3.6-35B-A3B as the "orchestrator" agent (it's just what I use for my daily driver), it's responsible for chatting with my wife about the menu, recipes and ingredients, and tells the smaller agent what to put in the cart with our brand preferences, dietary restrictions, etc. in mind.
I could probably drop the smaller Qwen at this point, but when I was first building this I was having an issue with search results and cart data filling up the main agent's context.
Nope! Not even once, which surprised me. We've done about 6 grocery runs with it now, and it's never been blocked by any anti-bot detection. Maybe it's because I'm signed into my Walmart account already, or because it's using Chrome's devtools protocol to control the browser instead of something like Puppeteer?
I was going to say that I'm using Apple's iCloud VPN, but I don't think Chrome actually uses that on Mac. I think you're right, it's probably my residential IP.
playwriter.dev will reuse your existing session or cookies I really like it over cdp when its stuff i need done from my regular chrome profile (you can also make separate profiles on chrome so the agent uses only that or smt like that)
I use them to find product fitting my specific desired parameters. Stores and price comparison websites often don't give you filters on all of the parameters that they posses and don't display all parameters on the list views.
I also tell my agents to remove annoyances from websites I browse, rearrange the content so that it's easier for me to view. For example when somebody publishes a table where they compare their newly released AI model to others I tell my agent to highlight highest result for each benchmark in every table on the page. I could do it myself with a bit of JS but why bother if agent can write it for me. I added a functionality to my agentic browser that lets the agent make userscripts for me that I can trigger with a push of a button.
I also ask agents whether the specific information is on the page that I'm currently browsing in language I don't understand (or just among the clutter).
Once I asked agent to put more than a dozen items into a cart for me (which names I pasted) because the ecommerce site didn't have convenient way of doing that.
So basically Grease Monkey on steroids + TD;DR;whaat?
Local Qwen3.6 is smart to do all that but I have option to switch to remote stronger models.
It’s usually incidental. Costco’s receipts aren’t easy to get. You go to a particular page listing them all and then for each date you click view receipt and look for the thing you want. You can’t really search because the thing will be labeled SHWR103 or something.
I didn’t “use an agent to find a receipt” in the sense that I purpose built one. I just asked my existing agent that I talk to on telegram by photographing the thing I wanted to know if we could return and while I changed the baby it chugged along and by the time we were ready to go it could tell me whether we did buy it at Costco and when so I know if I can return it.
- Apple Appstore Connect (gazillions of forms of metadata to release an app)
- AWS
- DigitalOcean
- Google Play Store
Whenever I dread logging in because I know the simple sounding task requires me to click through countless menus I use an agent browser. With confirmations of course. However, while the agent clicks through these (oftentimes dog slow) UIs I can do other things. Once it requires permission, I read, decide and act.
i've been tempted to use an agent to help me find non-horrible seats to see the 70mm imax version of the odyssey. but that's also the kind of thing that is heavily guarded by anti-bot detectors.
I often use it to browse sites to get web/app design inspiration, then make moodboards based off it and send it to me. saves me a lot of manual browsing time
Web scraping is the age old pre-LLM way to implement scratch-your-own-itch automations on the web. Anything you do manually with a web browser. Also used in corporate settings to interface with bad "enterprise" software, it's called RPA there. This kind of thing is the logical AI progression of those.
tldr; to workaround the lack (or shortcomings) of public m2m APIs in web apps
Comments
Can someone give me examples of where you use agents in your browser? I’ve heard executive leaders tout that “people use agents to buy things for them” but I haven’t actually seen that.
My wife really dislikes building up the shopping cart for our weekly grocery delivery, so I built an agent... thing with earendil's npm libs. It takes the menu my wife has decided on, confers with her about the ingredients (if it hasn't seen a recipe before), and then uses Chrome's devtools protocol to head to Walmart and add everything to the shopping cart.
It works fairly well and uses the local models I have running on my Mac Studio.
That is fantastic. Last I checked models capable of running on commodity (anything below a dedicated GPU rack) hardware were very lackluster.
Yeah, they're excellent. I use a small Qwen3-4B-Instruct as the "actor" agent which goes out to the website, searches for a batch of items, decides which best matches the criteria, and puts them in the cart. Then I have Qwen3.6-35B-A3B as the "orchestrator" agent (it's just what I use for my daily driver), it's responsible for chatting with my wife about the menu, recipes and ingredients, and tells the smaller agent what to put in the cart with our brand preferences, dietary restrictions, etc. in mind.
I could probably drop the smaller Qwen at this point, but when I was first building this I was having an issue with search results and cart data filling up the main agent's context.
Does walmart keep prompting you with "Press and hold" anti bot requests? I have a similar project and keep getting these on walmart.
Nope! Not even once, which surprised me. We've done about 6 grocery runs with it now, and it's never been blocked by any anti-bot detection. Maybe it's because I'm signed into my Walmart account already, or because it's using Chrome's devtools protocol to control the browser instead of something like Puppeteer?
You being authenticated is probably the big helper, yes.
it's trivial to detect vanilla CDP sessions.
There's many parameters in calculating a risk profile including having a residential IP number. I think you have that.
I was going to say that I'm using Apple's iCloud VPN, but I don't think Chrome actually uses that on Mac. I think you're right, it's probably my residential IP.
playwriter.dev will reuse your existing session or cookies I really like it over cdp when its stuff i need done from my regular chrome profile (you can also make separate profiles on chrome so the agent uses only that or smt like that)
I'm not exactly sure but playwriter highly likely will pollute the global name space of your existing session.
Very interesting!
I use them to find product fitting my specific desired parameters. Stores and price comparison websites often don't give you filters on all of the parameters that they posses and don't display all parameters on the list views.
I also tell my agents to remove annoyances from websites I browse, rearrange the content so that it's easier for me to view. For example when somebody publishes a table where they compare their newly released AI model to others I tell my agent to highlight highest result for each benchmark in every table on the page. I could do it myself with a bit of JS but why bother if agent can write it for me. I added a functionality to my agentic browser that lets the agent make userscripts for me that I can trigger with a push of a button.
I also ask agents whether the specific information is on the page that I'm currently browsing in language I don't understand (or just among the clutter).
Once I asked agent to put more than a dozen items into a cart for me (which names I pasted) because the ecommerce site didn't have convenient way of doing that.
So basically Grease Monkey on steroids + TD;DR;whaat?
Local Qwen3.6 is smart to do all that but I have option to switch to remote stronger models.
It’s usually incidental. Costco’s receipts aren’t easy to get. You go to a particular page listing them all and then for each date you click view receipt and look for the thing you want. You can’t really search because the thing will be labeled SHWR103 or something.
I didn’t “use an agent to find a receipt” in the sense that I purpose built one. I just asked my existing agent that I talk to on telegram by photographing the thing I wanted to know if we could return and while I changed the baby it chugged along and by the time we were ready to go it could tell me whether we did buy it at Costco and when so I know if I can return it.
For any horrible web administration UI:
- Apple Appstore Connect (gazillions of forms of metadata to release an app) - AWS - DigitalOcean - Google Play Store
Whenever I dread logging in because I know the simple sounding task requires me to click through countless menus I use an agent browser. With confirmations of course. However, while the agent clicks through these (oftentimes dog slow) UIs I can do other things. Once it requires permission, I read, decide and act.
i've been tempted to use an agent to help me find non-horrible seats to see the 70mm imax version of the odyssey. but that's also the kind of thing that is heavily guarded by anti-bot detectors.
I often use it to browse sites to get web/app design inspiration, then make moodboards based off it and send it to me. saves me a lot of manual browsing time
I would use them to solve captchas if that was a thing.
lol
Web scraping is the age old pre-LLM way to implement scratch-your-own-itch automations on the web. Anything you do manually with a web browser. Also used in corporate settings to interface with bad "enterprise" software, it's called RPA there. This kind of thing is the logical AI progression of those.
tldr; to workaround the lack (or shortcomings) of public m2m APIs in web apps