The Problem of Cumulative Advantage
"Cumulative advantage" is when small initial successes compound over time to create bigger later advantages. But, whether a particular ad gets a success early, or late, is partially random. So cumulative advantage can increase the randomness of an ad's success.
For example, let's say one ad sees an early success. The ad system could then estimate that it is a better ad, and show it in more competitive placements or lower the cost of each impression so it gets shown more often. This gives that ad more opportunities for further success, so it could outperform ads of the same quality.
Continuous Focusing Instead of Significance
February 15th, 2023
We use a different type of statistics (Bayesian) that gives you the probability each option is the best, both during and after your tests.
How we generate ads with your product
January 7th, 2023
We can include your product in generated photos; easily testing it in different contexts.
How our ad generation service works
January 6th, 2023
A walk through of the steps in our ad generation service.
How our testing service works
January 6th 2023
A quick walk-through of the process our testing service uses.
How to Improve Your Site with Coordinated Tests
September 29th 2022
Experiments that focus on the ideas you are testing, rather than the platform, can help you learn even faster.
Dynamic Creative vs Split Tests?
May 24th 2022
A quick overview of the differences between Split Testing and Dynamic Creative on Facebook. They have complimentary goals, so can accomplish great things together.
How are the probabilities calculated?
April 28th 2022
We help you understand your experiments, by carefully designing their structure and using a type of machine learning that allows easier interpretation.
How much should I spend testing new ads or audiences?
March 31th 2022
Happily, there is a technique that does a great job finding the best balance between testing new ads and using the best ones we've found.