Thanks, we think it's pretty cool too! And you've totally hit on why this is awesome - by automatically funneling users towards better-performing variations you can focus your resources on validating what matters and cut down on the deadweight loss associated with a traditional A/B. For more on this here's a recommended read: http://stevehanov.ca/blog/index.php?id=132?utm_medium=referr...
Right now auto-optimization only supports binary goal types. But we support A/B testing with Time and Quanity (numeric) goal types as well, so working on a solution to automate those types of experiments.
Comments
Thanks, we think it's pretty cool too! And you've totally hit on why this is awesome - by automatically funneling users towards better-performing variations you can focus your resources on validating what matters and cut down on the deadweight loss associated with a traditional A/B. For more on this here's a recommended read: http://stevehanov.ca/blog/index.php?id=132?utm_medium=referr...
Do you guys do any sort of logistic regression for optimizing numeric values? That would be really cool.
Right now auto-optimization only supports binary goal types. But we support A/B testing with Time and Quanity (numeric) goal types as well, so working on a solution to automate those types of experiments.