Comment on System Design for Entity ResolutionparentComments−rkwzOP2yData still needs to be cleaned ;)ML based linkage has evolved a lot over the past years, and a few people are trying out LLM based ones - https://arxiv.org/pdf/2403.06434v1−PaulHoule2yI'd assume you could train a cross encoder network likehttps://sbert.net/docs/cross_encoder/usage/usage.htmlto "predict" if two entities are the same?
Comments
Data still needs to be cleaned ;)
ML based linkage has evolved a lot over the past years, and a few people are trying out LLM based ones - https://arxiv.org/pdf/2403.06434v1
I'd assume you could train a cross encoder network like
https://sbert.net/docs/cross_encoder/usage/usage.html
to "predict" if two entities are the same?