Deciding Data

Mastering The Science of Marketing: Experimentation Processes

April 4th, 2023
Diagram of three experimentation processes

Marketing teams have to make three particularly impactful decisions. Sure, these are difficult. But, what makes them devious, is that the best answers keep changing. To adapt, many teams succeed with experimentation processes (and AI tools) that allow them to continuously check what works best.

The three decisions marketing teams need to make are:

  • Where do we run ads?
  • What ads do we run?
  • How much do we spend on them?

The answers to these questions often determine whether marketing, and sometimes the whole company, is profitable. But, the best-performing ad changes pretty quickly, sometimes even weekly. Oh, and to keep you on your toes the best-performing channel and ideal spending amount also change, but on different schedules.

To navigate through these swirling currents, many teams set up testing processes dedicated to each of these questions.

  • Mix Management: To help understand where to run ads
  • A Creative Process: To help consistently build high performing ads
  • A Scaling Process: To efficiently produce a meaningful amount of sales using your ads and channels.

Diagram of the three processes

Experimentation & AI Tools

When it is possible to run randomized experiments, it is often the best way to understand what causes what. So, experimentation is usually a key part of each of these processes.

How do we tell which channels are working the best? Let's run a test.

How do we know whether the color of the product matters to customers? Test it.

How do we tell whether a new bidding technique works better? ... Yup, also testing.

The thing about experiments, is they can be at once both tedious (how many variants are we creating?) and complex (wait, we need an integral of that function?). If it weren't for AI help, we'd probably just pretend these were simple questions that we can answer after a swing of whisky. With luck, your competitors are still pretending that.

To focus on answering one of these questions a system often needs to let go of answering the other two. Bid optimization systems don't give us many understandable insights about what product should be the focus of our next ad. Similarly, great creative feedback doesn't tell us the best place to run an ad.

Diagram of the three processes

Deciding Data, AI for the Creative Process

Deciding Data focuses on the creative process. It is built to provide feedback on what is working, and what features of an ad are most impactful. It supports the creative team to make hypotheses and iterate. This helps the team consistently make ads that resonate with your customers.

Deciding Data can help identify ads that are ready to scale, and even automatically hand them over to some scaling systems. But, Deciding Data doesn't manage the scaling. Scaling well involves navigating the quirky contours of each ad platform. Scaling systems have to optimize bidding, not step on the toes of the platform's learning systems, and limit ad fatigue. Because of this, scaling approaches are different between platforms. In fact, they are so specific to the ad platform that the platform's AI is often one of the best-performing scaling systems. Advantage+ Shopping campaigns on Meta and Performance Max campaigns on Google can often do pretty well.


Testing is a key part of systems to build great ads, run them in the right place, and spend the right amounts. Deciding Data is focused on doing a great job providing feedback on ads, and super-charging your creative process.

For AI tools that help with the other processes, we can make some great recommendations.

Deciding Data helps make ad testing easy and understandable. If you would like a tool to help understand your ads. We're happy to chat.