you could use some cheaper LLMs for this, even GPT-3.5turbo could excel at this or the moderations API (but that in my experience was more trained to obey the laws of USA, for example guns were okay to be asked about but are pretty much illegal in my country). The simplest way is a blocklist of terms which was what we had previously used containing an insane number of terms beyond the regular abusive terms, but it would need some updating from time to time.
Comments
you could use some cheaper LLMs for this, even GPT-3.5turbo could excel at this or the moderations API (but that in my experience was more trained to obey the laws of USA, for example guns were okay to be asked about but are pretty much illegal in my country). The simplest way is a blocklist of terms which was what we had previously used containing an insane number of terms beyond the regular abusive terms, but it would need some updating from time to time.