This is just a quick post to let you know that the Peer Comparison Spreadsheet has been updated and now includes a statistical significance check—courtesy of Richard Fergie.
Peer Comparison Statistical Significance
Richard was kind enough to add both an Agresti-Coull and Wilson significance check to the spreadsheet making it a much more useful tool. Now, not only can you detect big differences in performance for keywords, search queries, etc. from their peers you can also see if the differences are significant.
For more information about the math that Richard used, please visit his website. And don’t forget to say thanks while you’re there.
Chad (Richard), thanks a lot for adding to Chad’s great spreadsheat.
However, I think any judgement on significance of the difference between CTRs (proportions in general) should take into consideration a confidence interval for the Peer CTR.
What do you think?