i'm pretty sure most content moderation strategies operate on a budget below what most LLMs would cost.
for any site hosting user-generated content, how efficiently you can run your moderation system is essentially your unique selling point - LLM is the baseline.
You may be shocked at how inexpensive some of the LLMs are these days.
Google Gemini 1.5 Flash 8B charges $0.04/million input tokens and $0.15/million output tokens.
If a piece of content that needs to be moderated is 1,000 tokens (that's pretty long!) and you expect a 10 token return it will cost you 0.0039 cents - that's not a dollar amount, that's less than a 250th of a single cent.
So 1 cent will moderate 250 items of content. $1 will moderate 25,000 items of content.
you could use some cheaper LLMs for this, even GPT-3.5turbo could excel at this or the moderations API (but that in my experience was more trained to obey the laws of USA, for example guns were okay to be asked about but are pretty much illegal in my country). The simplest way is a blocklist of terms which was what we had previously used containing an insane number of terms beyond the regular abusive terms, but it would need some updating from time to time.
Comments
i'm pretty sure most content moderation strategies operate on a budget below what most LLMs would cost.
for any site hosting user-generated content, how efficiently you can run your moderation system is essentially your unique selling point - LLM is the baseline.
You may be shocked at how inexpensive some of the LLMs are these days.
Google Gemini 1.5 Flash 8B charges $0.04/million input tokens and $0.15/million output tokens.
If a piece of content that needs to be moderated is 1,000 tokens (that's pretty long!) and you expect a 10 token return it will cost you 0.0039 cents - that's not a dollar amount, that's less than a 250th of a single cent.
So 1 cent will moderate 250 items of content. $1 will moderate 25,000 items of content.
LLMs are dirt cheap. You can play around with pricing across different models using my calculator here: https://tools.simonwillison.net/llm-prices
you could use some cheaper LLMs for this, even GPT-3.5turbo could excel at this or the moderations API (but that in my experience was more trained to obey the laws of USA, for example guns were okay to be asked about but are pretty much illegal in my country). The simplest way is a blocklist of terms which was what we had previously used containing an insane number of terms beyond the regular abusive terms, but it would need some updating from time to time.
The alternative is paying a person…