Show HN: Bayesian AB-test analysis only needs 4 numbers

Hey (:)

You'd be surprised how many of the (quite talented) data analysts I've worked with don't truly understand the beautiful simplicity of Bayesian AB testing.

When AB testing, most metrics we care about are binomial (i.e., did or didn't happen -> 1 or 0). Think "User clicked login" or "User upgraded to Premium".

This means if you want to build an API for statistically analyzing these experiments all you need as inputs are 4 numbers (per metric): - Control: Total users, and users who "did" - Variant: Total users and users who "did"

Then you can reconstruct the "observation array" on the other side of the API, statistically compare your control to your variant, and send the results back!

Right now, if you have a way to pull those 4 numbers from Amplitude, or your database, you can run a Bayesian analysis using the analyzer i built.

Have fun!

rate limits do apply, product translation sold separately