What is statistical significance?

Statistical significance means the observed difference between A and B is unlikely to be due to random chance. Typically, a p-value below 0.05 (95% confidence) is considered significant.

How long should I run an A/B test?

Run until you reach statistical significance OR your predetermined sample size. Generally, 1-4 weeks minimum. Never stop a test early just because results look promising.

The p-value is the probability of seeing the observed difference (or more extreme) if there were truly no difference. A p-value of 0.05 means a 5% chance the result is due to randomness.

What sample size do I need?

It depends on your baseline conversion rate and the minimum detectable effect. Generally, 1,000+ visitors per variant is a good starting point. Lower baseline rates need more samples.

Can I test more than 2 variants?

Yes (A/B/C testing), but you need to adjust for multiple comparisons to avoid false positives. The Bonferroni correction divides the significance level by the number of comparisons.

A/B Test Calculator

Check if your A/B test results are statistically significant.

Variant A (Control)

Visitors

visitors

Conversions

conversions

Variant B (Test)

Visitors

visitors

Conversions

conversions

Confidence Level

Usually 95% or 99%.

Coursera Plus

Unlimited access to 7,000+ world-class courses, hands-on projects, and job-ready certificate programs.

Start Learning

⏳ Not Yet Significant

Uplift

+0.00%

B outperforms A

0.00%

Confidence

0.0000

p-value

Control (A)

0.00%conversion rate

Variant (B)

0.00%conversion rate

Not enough evidence yet

The p-value of 0.0000 is above the 5% threshold. Continue collecting data.

Documentation

How It Works

This calculator uses a two-proportion z-test to determine if the difference in conversion rates between A and B is statistically significant.

Formula

Frequently Asked Questions

About the A/B Test Calculator

What it calculates

This a/b test calculator measures marketing and advertising performance metrics. Check if A/B test results are statistically significant.

When to use it

Use when evaluating campaign ROI, ad spend efficiency, or funnel conversion performance.

Example

Adjust the inputs above to model your specific scenario with the A/B Test Calculator.

Explore More

A/B Test Calculator

Coursera Plus

+0.00%

0.00%

0.0000

How It Works

Formula

About the A/B Test Calculator

Related Calculators

Email Marketing ROI Calculator

Engagement Rate Calculator

Influencer Earnings Estimator

YouTube Money Calculator

CAC Calculator

Conversion Rate Calculator