This is absolutely fascinating. I am not sure if its the case that A/B testing is at fault but rather poor understanding of human nature and player dynamics. Thank you for sharing!
It's hard to be definitive, but I can see how if you were A/B testing then all changes would have indicated improvements of all KPIs, yet as a whole cumulatively over long term they led to downfall. It just might not be possible to glean ultimate long term human-nature's response from simple week-over-week active users count. Kind of like falling for local maxima and then falling to global low.
Comments
This is absolutely fascinating. I am not sure if its the case that A/B testing is at fault but rather poor understanding of human nature and player dynamics. Thank you for sharing!
It's hard to be definitive, but I can see how if you were A/B testing then all changes would have indicated improvements of all KPIs, yet as a whole cumulatively over long term they led to downfall. It just might not be possible to glean ultimate long term human-nature's response from simple week-over-week active users count. Kind of like falling for local maxima and then falling to global low.