When it was discussed 2 years ago, I teased out[0] (with a dose of cynical attitude) some key points about structural problems with A/B testing tools; problems, which navigate people away from scientific methods, and into the realm of easy-exploitable bullshit artistry.
I never had an opportunity to use or check out VWO, so I'd like to ask you: does your tool structurally helps users create sound statistical experiments? Can you share something about how you do it?
Comments
How much of A/B testing being done is actually scientific?
The article touches on that a little bit, but I'm reminded of this gem from 2014:
http://blog.sumall.com/journal/optimizely-got-me-fired.html
When it was discussed 2 years ago, I teased out[0] (with a dose of cynical attitude) some key points about structural problems with A/B testing tools; problems, which navigate people away from scientific methods, and into the realm of easy-exploitable bullshit artistry.
I never had an opportunity to use or check out VWO, so I'd like to ask you: does your tool structurally helps users create sound statistical experiments? Can you share something about how you do it?
--
[0] - https://news.ycombinator.com/item?id=10873226
Yes, check out the methodology we use: https://vwo.com/blog/smartstats-testing-for-truth/