With the rise of highly effective black box models, there's been a rise in interest towards interpretability for machine learning. This is an area I'm excited about, so I did a dive into the existing research.
I found a few surprising things (lack of user studies and some explanations even "working" on randomly initialized models), and I wanted to share a glimpse into the field for others who might be interested.
There's definitely a lot more room to explore, be it from the usability standpoint or from the more technical standpoint. Hopefully this can be an accessible jumping off point for others trying to enter the conversation.
Comments
With the rise of highly effective black box models, there's been a rise in interest towards interpretability for machine learning. This is an area I'm excited about, so I did a dive into the existing research.
I found a few surprising things (lack of user studies and some explanations even "working" on randomly initialized models), and I wanted to share a glimpse into the field for others who might be interested.
There's definitely a lot more room to explore, be it from the usability standpoint or from the more technical standpoint. Hopefully this can be an accessible jumping off point for others trying to enter the conversation.