Treat it like somebody grading an essay. It is either pass/fail or you can grade from 0 to 100 though you probably want to develop some kind of scoring rules for the latter.
I know a lot about evaluating classification but I am seeing people struggle w( evaluating text genersators; why don’t you look up my profile, send me an email, and we can talk more.
Comments
Causal text generation. I evaluated it on my own internal dataset but real world data shifts over time.
How do you get a number for text generation?
Exactly my question. I just scanned over the inputs and outputs and compared how "useful" they were.
Treat it like somebody grading an essay. It is either pass/fail or you can grade from 0 to 100 though you probably want to develop some kind of scoring rules for the latter.
I know a lot about evaluating classification but I am seeing people struggle w( evaluating text genersators; why don’t you look up my profile, send me an email, and we can talk more.