ChatGPT absolutely can visit websites provided by users. I proved that to myself just now by giving it a URL to a site I own, telling it to "Summarize URL", tailing the log file and confirming that the summary it returned included information that I had published on that page seconds before I prompted it: https://chat.openai.com/share/34f47777-59c8-47b0-9220-a1ce9b...
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
Generally, if you see it say "Browsing..." it means it's either running searches on the Bing index or actually visiting pages. I don't think it's possible to tell which though, which is annoying.
Not sure if this is still possible, but before they introduced the GPT marketplace and were doing the plugins thing, it was possible to prompt inject ChatGPT by including a payload in the HTML of a site you control and asking it to summarise the page. I'm pretty amused at the GPs hope that something like this wouldn't be possible because it would create security concerns, because, yes, exactly! We're going to see a lot of SSRFs, file read vulns, etc. as LLM assistants are baked into more platforms. These things aren't magic and the same approach you would take to installing a WordPress plugin or some other third-party library must be used here. That said, given the attack surface plugins and libs continue to add to web apps, I'm not optimistic about this being solved before it becomes a problem.
"I'm sorry, but I can't browse external websites, including the one you provided. However, if you share some information or specific details from the page, I'd be happy to help you summarize or discuss them!"
Comments
ChatGPT absolutely can visit websites provided by users. I proved that to myself just now by giving it a URL to a site I own, telling it to "Summarize URL", tailing the log file and confirming that the summary it returned included information that I had published on that page seconds before I prompted it: https://chat.openai.com/share/34f47777-59c8-47b0-9220-a1ce9b...
But... it's also often VERY resistant to visiting pages. It makes judgement calls based on the URL you pass it - it refused to visit a PostBin URL I set up just now, at least at first: https://chat.openai.com/share/dcacf056-4509-40e7-bc54-6b9690...
Eventually it did visit it - if you hit https://www.toptal.com/developers/postbin/b/1709008827487-07... you may see the request from it (if it's not expired), which included the following user-agent:
Generally, if you see it say "Browsing..." it means it's either running searches on the Bing index or actually visiting pages. I don't think it's possible to tell which though, which is annoying.Before they introduced ChatGPT browsing mode it was unable to browse the web but frequently acted like it could, causing much confusion. I wrote about that here: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...
I stand corrected. Thank you. I had not managed to trigger this behavior before, and falsely believed my experience was universal.
Not sure if this is still possible, but before they introduced the GPT marketplace and were doing the plugins thing, it was possible to prompt inject ChatGPT by including a payload in the HTML of a site you control and asking it to summarise the page. I'm pretty amused at the GPs hope that something like this wouldn't be possible because it would create security concerns, because, yes, exactly! We're going to see a lot of SSRFs, file read vulns, etc. as LLM assistants are baked into more platforms. These things aren't magic and the same approach you would take to installing a WordPress plugin or some other third-party library must be used here. That said, given the attack surface plugins and libs continue to add to web apps, I'm not optimistic about this being solved before it becomes a problem.
Great follow-up. It is odd to have this inconsistency in behaviours, and a bit scammy from openai, as these are paid features.
There's "Browser GPT" in the GPT Store which uses sider.ai to search the net and tends to do it liberally.
I think its willingness to use tools/functions has to do with the descriptions and system prompt more than anything (measurably) intrinsic to the LLM
"I'm sorry, but I can't browse external websites, including the one you provided. However, if you share some information or specific details from the page, I'd be happy to help you summarize or discuss them!"
3.5. Refuses 100% of all requests.
3.5 can't do this. It's a feature of paid ChatGPT running GPT-4.