But I think this is also the hardest task of a PM, so I am skeptical. There is a lot less learning and training material for an AI to use (compared to for example writing code) - so it is no surprise that AI in its current state does often not lead to great results.
Just to wrap this back, I suspect there's a way to process these sessions really quickly to figure out what the user trends are in ~real-time. This way you can do the same thing you're already doing, but with much more context.
Adding another two channels, with these channels being:
1. A weighted map of stats for all of the sessions at once.
Weighted by heuristics you choose or by good defaults.
2. Reports detailing all the natural problem groups the sessions fall into.
With breadcrumb trails available for deep-dives.
Most importantly I think there's a way to do this without having to rely on llms at all; by modeling the whole thing as a set of graph problems.
that's interesting to hear, thanks! and it does confirm my suspicions.
If you don't mind me asking, when you watch sessions, do you have a better way to prioritize the session you watch than just picking at random?
For example, do you have some way to pull out a bunch of session groups from the data automatically so you only have to watch one session in the group to know what the problems were for all of the sessions in that group?
Sessions are just one "channel" to build up an understanding of the problem, so I try to prioritize the ones where I have prior indication that they might be interesting (churn, complaints in a conversation or feedback via some other channel). But sometimes it can also be interesting to just "surf" through some random ones.
I'm interpreting from this that you have more data than you have time to process, so you go after the signals in order of how close they are to the user's complaints (user convos, etc).
And this implies to me that your ideal scenario is one in which the amount of data coming in from user convos, surveys, complaints, tickets, etc is equal or greater than the amount of time you have as a team to process it, such that you can focus on that and keep yourself productively at capacity.
But what if the amount of such high-priority signals is much higher than what you can deal with? Is it worth clustering that to get a smaller list of actionable trends?
Furthermore, if this is the highest quality data, is there even any need to go in and 'process' ALL of the sessions and bin them regardless of their high-priority signal status? Am I reading you right?
Comments
I would find it _very_ valuable.
But I think this is also the hardest task of a PM, so I am skeptical. There is a lot less learning and training material for an AI to use (compared to for example writing code) - so it is no surprise that AI in its current state does often not lead to great results.
Thanks again for your feedback!
Just to wrap this back, I suspect there's a way to process these sessions really quickly to figure out what the user trends are in ~real-time. This way you can do the same thing you're already doing, but with much more context.
Adding another two channels, with these channels being: 1. A weighted map of stats for all of the sessions at once. Weighted by heuristics you choose or by good defaults. 2. Reports detailing all the natural problem groups the sessions fall into. With breadcrumb trails available for deep-dives.
Most importantly I think there's a way to do this without having to rely on llms at all; by modeling the whole thing as a set of graph problems.
If this sounds reasonable to you lmk.
Have a good one!
that's interesting to hear, thanks! and it does confirm my suspicions.
If you don't mind me asking, when you watch sessions, do you have a better way to prioritize the session you watch than just picking at random?
For example, do you have some way to pull out a bunch of session groups from the data automatically so you only have to watch one session in the group to know what the problems were for all of the sessions in that group?
Sessions are just one "channel" to build up an understanding of the problem, so I try to prioritize the ones where I have prior indication that they might be interesting (churn, complaints in a conversation or feedback via some other channel). But sometimes it can also be interesting to just "surf" through some random ones.
I'm interpreting from this that you have more data than you have time to process, so you go after the signals in order of how close they are to the user's complaints (user convos, etc).
And this implies to me that your ideal scenario is one in which the amount of data coming in from user convos, surveys, complaints, tickets, etc is equal or greater than the amount of time you have as a team to process it, such that you can focus on that and keep yourself productively at capacity.
But what if the amount of such high-priority signals is much higher than what you can deal with? Is it worth clustering that to get a smaller list of actionable trends?
Furthermore, if this is the highest quality data, is there even any need to go in and 'process' ALL of the sessions and bin them regardless of their high-priority signal status? Am I reading you right?