We were using Optimizely until we didn't qualify for the lowest tier paid plan. Not enough impressions?
We knew at our traffic tier, Optimizely experiment analysis was not going to be statically robust. We were going to have our data person do the analysis with different techniques to deal with the smaller sample size. What we really wanted was the interface for a non-technical person to create the variation and Optimizely to manage which variation people saw.
Instead of letting us continue, our account was canceled. In the next months, when we do have the traffic, we are not going to consider them. Not impressed.
We were in a similar spot. We were on their (then) silver plan, but by the time we were ready to upgrade to gold, there was no gold. And no silver lining (pun intended). Only Enterprise. (see [0] for more details)
That was the reason I ended up creating Alephbet[1], an open-source JS A/B testing framework you can use with lots of different backends. And later Gimel[1], which is a backend that runs on AWS Lambda and Redis.
For developers, I think Alephbet and Gimel are a great free and open source alternative to commercial tools. We're running it in production for a couple of years now and our monthly AWS bills are $peanuts/month. It's NOT for marketers or non-technical users though.
You are welcome to use VWO. I can help you set up a trial account with us, and I usually hang around on HN to answer any of your questions. We have a comparison page here: https://vwo.com/comparison/optimizely/
I was disappointed that YCombinator funded Optimizely given that they were expressly copying the idea of a small, single-founder who was sharing his business and experience on these boards. As an observer admittedly without knowledge of who made that funding decision and why, it still seems the most unethical thing YC has ever done.
Comments
We were using Optimizely until we didn't qualify for the lowest tier paid plan. Not enough impressions?
We knew at our traffic tier, Optimizely experiment analysis was not going to be statically robust. We were going to have our data person do the analysis with different techniques to deal with the smaller sample size. What we really wanted was the interface for a non-technical person to create the variation and Optimizely to manage which variation people saw.
Instead of letting us continue, our account was canceled. In the next months, when we do have the traffic, we are not going to consider them. Not impressed.
We were in a similar spot. We were on their (then) silver plan, but by the time we were ready to upgrade to gold, there was no gold. And no silver lining (pun intended). Only Enterprise. (see [0] for more details)
That was the reason I ended up creating Alephbet[1], an open-source JS A/B testing framework you can use with lots of different backends. And later Gimel[1], which is a backend that runs on AWS Lambda and Redis.
For developers, I think Alephbet and Gimel are a great free and open source alternative to commercial tools. We're running it in production for a couple of years now and our monthly AWS bills are $peanuts/month. It's NOT for marketers or non-technical users though.
[0] http://blog.gingerlime.com/2015/alephbet-javascript-ab-test-...
[1] https://github.com/Alephbet/alephbet
[2] https://github.com/Alephbet/gimel
EDIT: formatting, typos
You are welcome to use VWO. I can help you set up a trial account with us, and I usually hang around on HN to answer any of your questions. We have a comparison page here: https://vwo.com/comparison/optimizely/
We are using VWO now! :D
These guys are the real deal.
I was disappointed that YCombinator funded Optimizely given that they were expressly copying the idea of a small, single-founder who was sharing his business and experience on these boards. As an observer admittedly without knowledge of who made that funding decision and why, it still seems the most unethical thing YC has ever done.