Are there any open source alternatives to Optimizely? If not, would an experienced SWE comment on an estimated difficulty level to get an MVP together?
With a ton of assumptions, mainly that you have an existing data or analytics pipeline, a simple in-house experimentation framework could take on the order of weeks rather than months to build.
There are no open source alternatives for the wysiwyg editor part of optimizely. But before you roll your own backend implementation have a look at https://facebook.github.io/planout/ .
(We had success in combining planout with a cms for the visual part. Marketing could do simple a/b tests. More complicated tests where done in the backend together with a developer.)
You need to be VERY careful using Google Analytics as a backend. After a certain threshold, they start sampling your data, and results will therefore become highly inaccurate.
Disclaimer: I also created an A/B test open-source framework with Google Analytics as (one potential) backend, but then realized it's sub-optimal. I wrote about it on [0].
Comments
Are there any open source alternatives to Optimizely? If not, would an experienced SWE comment on an estimated difficulty level to get an MVP together?
For A/B tests implemented in backend code, I just use rand() to bucket a user, then store that bucket in redis.
After it's been live for a while, I pull the data from redis then use this open-source calculator to view the results: https://thumbtack.github.io/abba/demo/abba.html
For anything more complicated (e.g. split test shared across multiple systems, A/B testing multi-step funnels) I fall back to Kissmetrics.
With a ton of assumptions, mainly that you have an existing data or analytics pipeline, a simple in-house experimentation framework could take on the order of weeks rather than months to build.
There are no open source alternatives for the wysiwyg editor part of optimizely. But before you roll your own backend implementation have a look at https://facebook.github.io/planout/ .
(We had success in combining planout with a cms for the visual part. Marketing could do simple a/b tests. More complicated tests where done in the backend together with a developer.)
Not a full alternative, but long ago I created this super-simple A/B testing tool you can use on top of Google Analytics - not back-end required!
https://github.com/danmaz74/ABalytics
You need to be VERY careful using Google Analytics as a backend. After a certain threshold, they start sampling your data, and results will therefore become highly inaccurate.
Disclaimer: I also created an A/B test open-source framework with Google Analytics as (one potential) backend, but then realized it's sub-optimal. I wrote about it on [0].
[0] http://blog.gingerlime.com/2016/a-scaleable-ab-testing-backe...
I already posted this on another thread[0]. Please check it out and happy to answer any questions or accept PRs.
[0] https://news.ycombinator.com/item?id=16357600
https://www.google.com/search?q=open+source+ab+testing
I had googled, but was looking for recommendations based on actual experimentation or use. Apologies that wasn’t clear.