This sounds just like the latest Michael Connolly Lincoln lawyer novel. Which made an interesting point I hadn't thought of: adults wrote the code for ChatGPT, not teenagers, and so the way it interacts with people is from an adults perspective.