In this video I explain how to use an excel spreadsheet for determining statistical significance for PPC ad testing. I have included a free download of the spreadsheet as well as links to the resources used to create the spreadsheet.

I used the ClickEquations blog for some inspiration and the Marketing Experiments blog, a Data Mining book, and statistically-inclined friend for some help.

Great work Chadd – this is extremely useful. There are far too many paid search text ad decisions made on incomplete data. I’m sure your tool will help people to make smarter decisions.

Thanks Craig, I’m glad you find it useful.

Hi Chad: Thanks for posting this. Just a quick question. I am trying to understand the formula for calculating “Needed difference”. Can you kindly explain how have you calculated that and what is the significance of this. I am trying to build a model to perform ad copy testing and finding out the statistically significant ad copies and their text. Thanks and I look forward for your reply.

Sorry Natasha, but I had a friend help me understand the math–I just built the spreadsheet.

I would recommend using the resources that I link to in the blog post, they do explain the math involved much better than I could.

Thanks Chad. But could you help me in understanding the significance of “Needed difference”? Does it mean that for the challenger ad to reach close to winner, it need to increase its CTR by the difference percentage? And if actual difference is greater than needed difference, it means we have a clear winner?

Thanks again

Yes, if the actual difference is greater than the needed difference you have a winner for the chosen confidence level.

Hope this helps.

Hi Chad:

I am still working with your excel and is quite helpful but 1 problem I am getting is, that the SEDP cell returns an error if the number os negative. Do you have any resoltuion for that?

What number is negative?

This is exactly what I was looking for thanks! Do you think it would work for display advertising in addition to PPC?

Hi Mike, thanks for stopping by. Yes it will work for testing display ads as well.

Thanks! This is very usefull.

Chad,

Nice work! Anyway this can be uploaded/shared with us in a Google Doc? Thanks!