Comment on KnowledgeNet: A Benchmark for Knowledge Base PopulationparentComments−nl6yIs this benchmark a good one?Any benchmark which reflects a task that humans do is a good one, unless it has specific weaknesses that a computer exploits.I'd use models that are written for this is my work, so I find it useful.I feel like this is true of any new benchmark.... give some smart folks a few months and they can now beat the task.In NLP work this use not to be the case. 5 years ago we were stuck at a local maximum.And this undervalues this task - this bridges the gap between unstructured and structured data. In many ways it is the holy grail for many tasks.
Comments
Is this benchmark a good one?
Any benchmark which reflects a task that humans do is a good one, unless it has specific weaknesses that a computer exploits.
I'd use models that are written for this is my work, so I find it useful.
I feel like this is true of any new benchmark.... give some smart folks a few months and they can now beat the task.
In NLP work this use not to be the case. 5 years ago we were stuck at a local maximum.
And this undervalues this task - this bridges the gap between unstructured and structured data. In many ways it is the holy grail for many tasks.