Control (A)
Variant (B)
Not Significant
Not enough evidence at the 95% confidence level. Consider running longer.
Control Rate
3.00%
Variant Rate
3.70%
Relative Lift
+23.3%
Absolute: +0.70%
p-value
0.3432
Z = 1.945
95% Confidence Interval for the Difference
0
-0.01%+0.70%1.41%
The interval crosses zero. The true effect could be positive, negative, or zero.
Statistical Power Achieved
49.4%Low power (< 80%). The test may not have had enough visitors to reliably detect this effect. Consider a larger sample.
Show the math
// Conversion rates
p_control = 3.0000%
p_variant = 3.7000%
p_control = 3.0000%
p_variant = 3.7000%
// Pooled proportion
p_pooled = 0.033500
p_pooled = 0.033500
// Pooled standard error
SE_pooled = sqrt(p_pooled * (1 - p_pooled) * (1/n1 + 1/n2))
SE_pooled = 0.003599
SE_pooled = sqrt(p_pooled * (1 - p_pooled) * (1/n1 + 1/n2))
SE_pooled = 0.003599
// Z-score
Z = (p_variant - p_control) / SE_pooled
Z = 1.945112
Z = (p_variant - p_control) / SE_pooled
Z = 1.945112
// Two-tailed p-value
p_value = 2 * (1 - Phi(|Z|))
p_value = 0.343221
p_value = 2 * (1 - Phi(|Z|))
p_value = 0.343221
// Confidence interval
SE_diff = sqrt(p1*(1-p1)/n1 + p2*(1-p2)/n2)
SE_diff = 0.003598
CI = (0.7000%) +/- 1.9600 * 0.003598
CI = [-0.0052%, 1.4052%]
SE_diff = sqrt(p1*(1-p1)/n1 + p2*(1-p2)/n2)
SE_diff = 0.003598
CI = (0.7000%) +/- 1.9600 * 0.003598
CI = [-0.0052%, 1.4052%]