It's been trivial to produce any kind of normally banned output with the API's themselves for years.

Producing actual erotica on your gmail-connected ChatGPT account seems like an idiot filter