Why would the scammers apply watermarking to their fraudulent data? Or do you expect that FTC somehow will make the open source generative models disappear worldwide?
I meant some watermarking that can’t be redone even if the weights are shared
I've been working a bunch on neural networks, and I'm quite convinced that this isn't possible (as in, not that we haven't built a tool for this yet, but that it's not possible to build such a tool and weights can always be redone) - I'm not wholly certain as I have been surprised before, but I'd need to see some evidence to even consider this as plausible. Fine-tuning can change a lot, adversarial examples can target very specific aspects of the model.
Furthermore, people do train models from scratch - even if 99.9% of models would include unremovable watermarks, all fraudsters need is for a single unmarked model to exist; and if it doesn't, they can fund the GPU-time to train one since they're running a high-revenue business, fraud has more money than academia.
Comments
Why would the scammers apply watermarking to their fraudulent data? Or do you expect that FTC somehow will make the open source generative models disappear worldwide?
I meant some watermarking that can’t be redone even if the weights are shared
It is still an open question but I think money will be dumped into this area, for protection of IPs and compliance
I've been working a bunch on neural networks, and I'm quite convinced that this isn't possible (as in, not that we haven't built a tool for this yet, but that it's not possible to build such a tool and weights can always be redone) - I'm not wholly certain as I have been surprised before, but I'd need to see some evidence to even consider this as plausible. Fine-tuning can change a lot, adversarial examples can target very specific aspects of the model.
Furthermore, people do train models from scratch - even if 99.9% of models would include unremovable watermarks, all fraudsters need is for a single unmarked model to exist; and if it doesn't, they can fund the GPU-time to train one since they're running a high-revenue business, fraud has more money than academia.