ABExperiment Types - Different design

How to design Experiments

Classic A/B Experiment - there is control A and a treatment (varience) B. This is most commonly usd method.

Leap Experiment - It is variation of above. Multiple changs/treatment are added together to test whether newer version provide lift. It reduce experimentation time for each change. It has a downside that you do not know which change is contributing how much to lift?

Exploratory Cascade ExperimentIn this method one design multiple variations. Each variation has a single change. The variations build on top of each other and this allows for comparisons across set. (Some teams call each set a flight). It is useful when you have very large traffic and one can use it to try out multiple solutions or simply explore what users like.

Exploratory Leap Experiment - In this we explore multiple different combination to determine which change bring benefit.

Reverse Experiment - This is similar to above. Here you explictly remove feature to find its effect tect Example : remove price from webpage and see it if it drop conversion.

Intensity Experiment - This type of experiment is useful if hypotheis is that intensity of treatment will impact outcome. There are scenario where a variable has a linear impact on outcome (Example : 1 testimonial, comments on post might be good, 2 even better, 3 even, 100+ is awesome