Statistical Power Calculation of an A/B test result in Excel


Notice: Trying to access array offset on value of type null in /bitnami/wordpress/wp-content/themes/echologyx/single-post.php on line 25

Notice: Trying to access array offset on value of type null in /bitnami/wordpress/wp-content/themes/echologyx/single-post.php on line 25

A friend of mine has recently asked if I can help him with creating an Excel file to calculate the power of A/B test results. I found one calculator online here which I think is pretty sleek but the only problem is, it is online. So if you want to do your own calculation using your excel file, you will need to know the maths behind it.

Rather than getting into the details of what the statistical power is, in simple terms, the statistical power determines the possibility of Type II error (or false negative) within the result that you are seeing from the test.

I have created this Excel file which follows the same calculation that has been used by the AB Test Guide (https://abtestguide.com/calc/) to calculate the statistical power of any A/B test.

Instruction to use this excel file to calculate statistical power is pretty self explanatory:

  • Select whether the test is 1-Tail or 2-Tail (in most cases, if you are running A/B tests using major tools that are in the market – it should be 2-Tail)
  • Select the confidence level (90% / 95% / 99%)
  • Add the visitor numbers for your control and variation
  • Add the conversion numbers (NOT Conversion Rate) for your control and variation

That’s it! It will show the power of the test – in other words – the possibility of Type II errors (the lack of it).

Download Statistical Power Calculation of AB Test Results- EchoLogyx

Don’t just take our word for it…

Don’t just take our word for it…

Customers’ Talking

Our customers are our biggest advocates. That’s why we let them do the talking.

Initially, when working with EchoLogyx, we ran a few test experiments to see how they perform. They did really well, QA was really positive. And then we sort of stepped it up very quickly from there. We are producing 12-15 tests a month. Communication is absolutely great. I can contact the team at any point in the day even after hours they are still responding to me. Since working with EchoLogyx, we have never had an issue with a test that has gone live.

Let's Have a chat?

If you want to find out a bit more, just get in touch. We love a chinwag, and we'd love to help you out.

Contact Us