|
|
Exclusion-Restriction in A/B Testing
Exclusion-restriction refers to the principle that in an A/B test, the control group and the experimental group should be mutually exclusive and collectively exhaustive. This means that each participant should only be included in one group and that all possible participants should be included in one of the groups.
Exclusion-restriction can impact A/B testing on web and in mobile in several ways:
| Impact |
Description |
| Sample Size |
If exclusion-restriction is not properly enforced, it can lead to overlap between the control and experimental groups, resulting in a smaller sample size and less reliable results. |
| Bias |
If certain participants are excluded from one of the groups, it can introduce bias into the results. For example, if a certain demographic is excluded from the experimental group, it may not accurately represent the population as a whole. |
| Validity |
If exclusion-restriction is not properly enforced, it can lead to invalid results. For example, if participants are able to switch between the control and experimental groups, it can invalidate the results. |
|
|
|