Not OP, but parent of multiple school-age kids and both:
1. You're 100% right, there are privacy concerns.
2. I don't know if they could possibly be worse than the majority of school districts (including my kids) running directly off of Google's Education system (Chromebooks, Google Docs, Gmail etc.).
You can generally register your child under an assumed name - at least, that is possible in my area. Families can choose this option if there is any threat to the security of their child.
It's important to differentiate concern(a feeling) from choosing to upload or not. In the calculus of benefits and risks, The feeling of concern(potentially leaking PII/health information) may be outweighed by the benefit in education. Even if someone is concerned, they may still see the positives outweigh the risks. It's a subjective decision at the end of the day.
I should have clarified that I used Adobe Acrobat to redact his personal identifiers from the report before uploading it to Claude. I generally also prompt using fake names. It's not perfect, but it's better than nothing.
And, on another note, this may be foolish, but I generally trust well funded organizations like Anthropic and OpenAI on the assumption that they have everything to lose if they leak private information from their paid users. Anthropic has a comprehensive and thoughtful privacy policy (https://www.anthropic.com/legal/privacy), which specifies they do not use your data to train their models, other than to refine models used for trust and safety:
"We will not use your Inputs or Outputs to train our models, unless: (1) your conversations are flagged for Trust & Safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Acceptable Use Policy, including training models for use by our Trust and Safety team, consistent with Anthropic’s safety mission), or (2) you’ve explicitly reported the materials to us (for example via our feedback mechanisms), or (3) by otherwise explicitly opting in to training."
As for defending against a data breach, Anthropic hired a former Google engineer, Jason Clinton, as CISO. I couldn't find much information about the relevant experience at Google that may have made him a good candidate for this role, but people with a key role in security at large organizations often don't advertise this fact on their LinkedIn profiles as it makes them a target. Once you're the CISO, the target appears, but that's what the big money is for.
Thanks for the vote of confidence. I led the Chrome Infrastructure Security Team hardening for insider risk and generally defending against APTs for the last 3 years at Google. Before that, I was on the Payments Security Team defending PII and SPII data up and down the stack. Indeed, I and the company take this very seriously. We're racing as fast as we can to defend against the run-of-the-mill opportunistic attackers but also APTs. We've ramped the securtiy team over the last year from 4 to 35 people. I'm still hiring, though!
Comments
Were yout not concerned about the privacy implications of uploading your child's sensitive halth data to a private LLM?
Not OP, but parent of multiple school-age kids and both:
1. You're 100% right, there are privacy concerns.
2. I don't know if they could possibly be worse than the majority of school districts (including my kids) running directly off of Google's Education system (Chromebooks, Google Docs, Gmail etc.).
Can you opt out? Are there privacy-friendly schools?
Could you enroll your child under a fake name? How messed up would they think that is :D
You can generally register your child under an assumed name - at least, that is possible in my area. Families can choose this option if there is any threat to the security of their child.
It's important to differentiate concern(a feeling) from choosing to upload or not. In the calculus of benefits and risks, The feeling of concern(potentially leaking PII/health information) may be outweighed by the benefit in education. Even if someone is concerned, they may still see the positives outweigh the risks. It's a subjective decision at the end of the day.
I should have clarified that I used Adobe Acrobat to redact his personal identifiers from the report before uploading it to Claude. I generally also prompt using fake names. It's not perfect, but it's better than nothing.
And, on another note, this may be foolish, but I generally trust well funded organizations like Anthropic and OpenAI on the assumption that they have everything to lose if they leak private information from their paid users. Anthropic has a comprehensive and thoughtful privacy policy (https://www.anthropic.com/legal/privacy), which specifies they do not use your data to train their models, other than to refine models used for trust and safety:
"We will not use your Inputs or Outputs to train our models, unless: (1) your conversations are flagged for Trust & Safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Acceptable Use Policy, including training models for use by our Trust and Safety team, consistent with Anthropic’s safety mission), or (2) you’ve explicitly reported the materials to us (for example via our feedback mechanisms), or (3) by otherwise explicitly opting in to training."
As for defending against a data breach, Anthropic hired a former Google engineer, Jason Clinton, as CISO. I couldn't find much information about the relevant experience at Google that may have made him a good candidate for this role, but people with a key role in security at large organizations often don't advertise this fact on their LinkedIn profiles as it makes them a target. Once you're the CISO, the target appears, but that's what the big money is for.
Thanks for the vote of confidence. I led the Chrome Infrastructure Security Team hardening for insider risk and generally defending against APTs for the last 3 years at Google. Before that, I was on the Payments Security Team defending PII and SPII data up and down the stack. Indeed, I and the company take this very seriously. We're racing as fast as we can to defend against the run-of-the-mill opportunistic attackers but also APTs. We've ramped the securtiy team over the last year from 4 to 35 people. I'm still hiring, though!
wink wink