In all likelyhood it will be an (neural net?) AI, and then it's hard to get insight into why decisions were made.
And even if they could explain it you, it might be spurious correlations that were picked up.
There is a huge field of interpretable/fair AI and these types of questions arise in much more serious instances (e.g. where people in prison aren't given parole due an AI "assisted" decision). The state of the art seems to be that there are no easy answers as soon as you start questioning a decision by a modern AI system (or want introspection). You can only hope to not be part of the 5% of the cases where the decision is bad.
Comments
Does a consumer have any insight into what the model does/doesn't like about an IP, or is it a black box?
I'm wondering how they could contextualize a score for their use case, or how I as the IP owner know what to fix to raise the score.
In all likelyhood it will be an (neural net?) AI, and then it's hard to get insight into why decisions were made.
And even if they could explain it you, it might be spurious correlations that were picked up.
There is a huge field of interpretable/fair AI and these types of questions arise in much more serious instances (e.g. where people in prison aren't given parole due an AI "assisted" decision). The state of the art seems to be that there are no easy answers as soon as you start questioning a decision by a modern AI system (or want introspection). You can only hope to not be part of the 5% of the cases where the decision is bad.