How to Improve Your Site with Coordinated Tests
Running experiments (A/B tests) on your website is a great way to improve the site. But, you can go even further, by adjusting where you test creative ideas. If you try them first where you have lots of traffic (often ads), you can test many more ideas. Then you can take the winners and refine them on the site, giving your website tests a notable head start.
Standard User Flow
Customers often first learn about a product from an ad, if it speaks to them, they move onto the site to potentially take an action (purchase, newsletter, etc.).
The standard flow often looks something like this:
The content used for the ads and site matters tremendously to how many people move through this flow. In a recent study, Nielson found that the creative is the single element with the largest impact on sales lift.
To refine the creative, it can really help to have feedback from your customers. Data-driven experiments help you collect that feedback, and improve your content.
More Traffic Means More Testing
The more traffic an experiment has, the more it can tell you about what works. But, traffic usually costs money, so one wants to use it well.
In general, we see more traffic higher in the user flow/funnel. Lots of people see the ads, but only a small fraction of them make it to the checkout pages. This means we can test many more creative ideas in ads than we can on the payment page.
To really understand what works, we can coordinate tests at different points in this flow. This allows us to make discoveries quickly, even though some parts of our site may only have limited traffic.
We can try out a wide range of ideas in ads, and understand which have the greatest potential. Then we can use these lessons to inform what we test on our site. This gives us a big head start in optimizing our site, so we can find great content to use.
Virtuous Cycle
When we find an idea that works well at driving traffic to the site, we can then test site changes on that traffic. This gives us better insights into what works well.
It also helps that more traffic makes our site tests run faster.
Upstream Benefits
When we find better content for our site, there are benefits that can flow upstream. For example, in ad auctions the Facebook system considers how often a user takes a later action (like a purchase). If users are more likely to take an action, Facebook effectively lowers the cost of ads, making customer acquisition even more efficient.
Coordinated Experiments
Coordinating the A/B tests that you run throughout your traffic funnel can help you quickly discover improvements. It lets you try many ideas quickly where you have lots of traffic (usually ads), and refine them where it matters most to the purchase (usually your site).