Struggling with Ads
A few years ago, I was at a holiday party and struck up a conversation with my friend Sarah about marketing. Eventually, we got to talking about SEM, and she mentioned that her company was struggling with getting more traffic for their ads. The company did lead generation for auto shops, and advertised for hundreds of them. We chatted some more, and I told her that SEM was an area I knew a fair bit about, and she invited me to her office to meet with their team to discuss what they were doing. I met with the CTO and CMO to discuss their strategy.
Sarah, who was a marketing manager, had created a few ads, and the company was achieving mediocre click-thru rate (aka “CTR” – how many people click on an ad divided by how many people view the ad). How could it be improved? The approach they’d taken, managing SEM through software that connected to an API, hadn’t really helped. I learned all of the shops received the same ads.
The fundamental problem
The fundamental problem, Sarah explained later over a beer, was the company didn’t have a great understanding of who its reSarahl customers were and what motivated them, specifically, to purchase auto repairs. They had paid for an expensive marketing study to create buyer personas, each of which had a general feeling about auto shops, but it had proven difficult to make that actionable with messaging.
As we paid and left, I wondered, could an artificial intelligence approach help the company find a better approach?
How could AI help?
Over the next few months, I started to think about how (human) marketers create ads. We identify 2 or 3 value propositions (the customer cares about price, about service, etc) and write ads for each one. All well and good. How could AI help with that? A program couldn’t write ads. It would have to know English words, and grammar, and concepts, and motivations of customers, etc etc. This is where the general field of AI had foundered, back in the 60s. Building a “general learning program” that has common sense and could reason about the world turned out to be very, very difficult. We still don’t have one. But was there a way that an AI could help write better ads, even if it didn’t know what the ads meant? Enter Genetic Algorithms.
In nature, animals that are better suited to their environment breed and pass their successful traits to their offspring. For example, the first long-necked giraffe had an advantage in getting more leaves from tall trees. That giraffe was healthier and had more children, who had the same long neck. Over time, all giraffes came to have long necks. Genetic algorithms use software to mimic this process of natural selection.
In a genetic algorithm, software creates a population of many “individuals” that are variations of something, and then applies a “fitness function” to see which ones work best. The winners are kept, and used to create the next generation of individuals. Importantly, the software program doesn’t have to have any actual understanding of what it’s doing. How would this work in practice?
As an example, imagine that you want to create a better airplane wing, with less drag. A computer might generate hundreds of designs, each slightly different. Each wing design could be tested in a simulated wind tunnel to see how much drag it created. The resulting data could be used by the program to generate new designs. But the software doesn’t need any real understanding of what drag is, or why you’d want it to be lower. It just has to know that the designs with the lowest drag win, and use those to create the next generation. Ok, back to ads.
In the case of SEM, our “individuals” were ads. We had a very clear fitness function — better ads, by definition, were those that had high click-thru rate (CTR). But what were the “traits” of ads? I thought on this for a long while, walking back and forth through the sunny streets of Soma, and the 2nd avenue park. Eventually, while walking around in my backyard and playing catch out with my friend’s dog, it came to me. The fundamental traits of ads were the purchasing motivations they spoke to.
Why do we buy?
Instead of trying to guess which motivation or set of motivations most influenced why someone decided to go to one auto shop over another, I could just us an AI approach to try every possible motivation. I could make a list of every possible reason someone might choose a auto shop (price, location, years in business, etc.), and then start to write little phrases that emphasized that motivation. These phrases would be the “traits” of an ad, that enabled them to compete better in their world, ie. get people to click on them. Importantly, there would be multiple phrases for each motivation, just like a genetic code for a giraffe’s spots might express itself in different patterns of spots.
Deciding on these motivations and creating the phrases around them is something that a human is uniquely well suited to do. Where AI comes into play is in doing the brute force work of creating hundreds of ads, and tracking how each one did.
The CTO and I designed software to reach into the bucket of phrases and then assemble them randomly into an ad that showcased one or more motivations, and then we assigned it to one auto shop’s SEM campaign to see how it did. Then after running the ads for a month or so, we pulled back all of the ads from all 800 campaigns. The results were suprising.
“And that’s why we do the experiment”
It turned out that there was a huge range in the click-thru of the ads, but our best few ads now had a click-thru rate of 1%, about double of what we started. Success! We looked at the winning ads, and found that they all shared one main motivation, which was not what we expected. We even found that some of the higher performing ads might mention the same thing twice, in two different ways. Some of the worst, most un-grammatical phrases that we threw in as random tests ended up being in the top twenty ads. And some of the motivations we were most interested in ended up doing poorly. But as the CTO and I mentioned to each other, “And that’s why we do the experiment”, because often our intuitions can be wrong. So now we had winners, how many ads should we devote to the winners, vs trying new experiments?
Imagine you’re in a vegas casino, surrounded by hundreds of slot machines. You have been told that there are some machines that are profitable, and a rare few that are extremely profitable. After playing for an hour or two, you discover that most slot machines don’t pay out, but you find one that seems to be profitable. Should you spend all your money on this machine, or devote some of your findings to continue to look for even better machines? In mathematics, this is known as the “N-Armed Bandit” problem. It has a direct parallel to our ad question. When we found ads that worked well, how often should we show those ads versus experimental ads which will usually fail, but sometimes might prove better than our current winners? There are a variety of solutions to this problem, but the one we settled on was to use proven ads with high CTRs most of the time, and to use a small amount of impressions for good ads that had limited data, and a smaller amount of time still for those that were completely experimental. I called these the Major, Minor and Farm Leagues. As ads accumulated data and were proven to be effective, they would be promoted to the next league up.
Over time, using a genetic algorithm and an ‘n-armed bandit’ approach, we were able to get very, very effective ads. Still, the implementation took quite a while, and there were many particular tweaks and modifications that we made to address specific Google Adwords issues.
However, a few broad principles did emerge that were helpful:
-Be exhaustive in the different purchasing motivations you consider.
-Don’t make assumptions, let software try out everything.
-Use a genetic algorithm to create your ads and decide which ones work best.