"AB Testing Alert: Beware of Spatial-Spillover!"

Spatial-Spillover

Spatial-spillover refers to the phenomenon where the effects of a treatment or intervention spill over from the treated unit to neighboring units. In the context of AB testing on the web and in mobile, spatial-spillover can occur when users in the treatment group interact with users in the control group, or when users in the control group are influenced by the changes made to the treatment group.

Impact on AB Testing Description
Increased Variability Spatial-spillover can increase the variability of the data, making it more difficult to detect significant differences between the treatment and control groups.
Contamination If users in the treatment group interact with users in the control group, or if users in the control group are influenced by the changes made to the treatment group, the results of the AB test may be contaminated.
Sample Size Spatial-spillover can affect the sample size required for an AB test. If the spillover effect is large, a larger sample size may be needed to detect significant differences between the treatment and control groups.
Validity Spatial-spillover can affect the validity of the results of an AB test. If the spillover effect is not accounted for, the results may not accurately reflect the true impact of the treatment or intervention.