Ask HN: statistical significance on group of Adwords keywords

I am trying to optimize my adwords campaign, and running into the 'statistical significance' question.

I read this great post from Paras Chopra here: http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-significance-calculator-spreadsheet-in-excel/

Though in my case I started my campaign with many keywords (about 400 keywords), and I now have the following:

Keyword group A (10 keywords - the "best"): 47 clicks, 16 conversions

Keyword group B (390 keywords - the rest): 244 clicks, 18 conversions

If I apply these data into the A/B Testing Significance calculator, I obtain a 99% confidence that A is better than B. But does it really work with a group of keywords, or is this calculation meant to be used only for strict "singular" test cases?

VorticonCmdr 13y

You can use it this way. But what does the data tell? Group A is definitely performing better than group B or better it is not a random coincidence that group A is better performing.

Comments