Deciding Data

The Problem of Cumulative Advantage

An abstract representation of growth

"Cumulative advantage" is when small initial successes compound over time to create bigger later advantages. But, whether a particular ad gets a success early, or late, is partially random. So cumulative advantage can increase the randomness of an ad's success.

For example, let's say one ad sees an early success. The ad system could then estimate that it is a better ad, and show it in more competitive placements or lower the cost of each impression so it gets shown more often. This gives that ad more opportunities for further success, so it could outperform ads of the same quality.

Continue reading...

Continuous Focusing Instead of Significance

February 15th, 2023
We use a different type of statistics (Bayesian) that gives you the probability each option is the best, both during and after your tests.
Continue reading...