A Very Strange AdWords Test

[abbottsys]

We spend lots of time carefully crafting and optimizing our AdWords ad copy - the headline, and 2 lines of body copy. We tweek, we tune, we measure. Finally we get a truly great ad.

I have several such ads that have performed superbly for years, pulling in major business. But am I absolutely sure it's because of my great ad copy? How do I know?

It occured to me that this could be measured just like anything else. So I'm running a variation against my best ad, but with bogus line copy. The variation reads:

Headline
Headline
Headline

Where "Headline" is the real headline from by best ad, but lines 2 and 3 just repeat this headline - i.e. there is no ad copy written for lines 2 and 3.

If my ad writing skills for lines 2 and 3 are so good then this ad should perform terribly compared with my *real* ad that I spent ages optimizing. Is ad copy for lines 2 and 3 actually needed? I'll let you know :)

[RedFly]

We spend lots of time carefully crafting and optimizing our AdWords ad copy - the headline, and 2 lines of body copy. We tweek, we tune, we measure. Finally we get a truly great ad.

I have several such ads that have performed superbly for years, pulling in major business. But am I absolutely sure it's because of my great ad copy? How do I know?

It occured to me that this could be measured just like anything else. So I'm running a variation against my best ad, but with bogus line copy. The variation reads:

Headline
Headline
Headline

Where "Headline" is the real headline from by best ad, but lines 2 and 3 just repeat this headline - i.e. there is no ad copy written for lines 2 and 3.

If my ad writing skills for lines 2 and 3 are so good then this ad should perform terribly compared with my *real* ad that I spent ages optimizing. Is ad copy for lines 2 and 3 actually needed? I'll let you know :)

Looking forward to the results. My faith in humanity depends on your new test ads performing terribly.

[glengara]

At least if they don't tank you'll be able to console yourself with your headline writing skills :-)

[abbottsys]

.... My faith in humanity depends on your new test ads performing terribly.
LOL! I know exactly what you mean. I'm keeping my fingers crossed for a total flop :)

[abbottsys]

At least if they don't tank you'll be able to console yourself with your headline writing skills :-)
Good point! I was careful to use the same headline in both ads, and it's a headline that took me ages to perfect.

BTW: The status so far.. the "bogus" ad just started getting clicks. No conversions yet.

[AdWordsRep]

A Very Strange AdWords Test

abbottsys, when I saw the subject line I instantly knew it would be you that started the thread. And I mean that as a compliment. ;)

I am just waiting for the book to be published:

"The Complete AdWords Tests of Abbottsys -- 101 Useful Tests by the King of AdWords Testing"

When you publish, I hope I can be one of those folks who writes a glowing blurb on the back cover. :)

AWR

[abbottsys]

...
"The Complete AdWords Tests of Abbottsys -- 101 Useful Tests by the King of AdWords Testing"...
AWR
LOL! What a great idea! I guess my need to test comes from my background, many years ago, as an experimental particle physicist in Europe. The guys I worked with were brutal. I mean, you could drop an apple and tell them it fell due to gravity and they would say, "oh really, are you sure, we'll test that hypothesis". LOL!

Meanwhile, prelim results from my latest ad test are either great or very disturbing, depending on your point of view :)

More to come..

[glengara]

Sounds ominous for Daves' faith in humanity......

Can you give any info on operational area, typical viewer profile?

[abbottsys]

PRELIM RESULTS

1) The bogus ad is performing exactly the same as the real ad in terms of #clicks and CTR.

2) Still to early to quote conversions because volume is not yet statistically significant. But, *if* all I was measuring was clicks and CTR then I would conclude both ads are performing identically. But I'm not, conversions rule, so I'll wait for those numbers to come in.

3) I have a couple of theories for the above, but I'll hold off drawing conclusions until all numbers are in.

BTW: we're in the medical field, generating leads for a high ticket elective medical procedure (no, not plastic surgery.)

[abbottsys]

RESULTS

Good grief!

1) The bogus ad has matched the real ad in terms of clicks, CTR and conversions. In short, its performance is superb.

This is a surprise, and *may* have implications for all AdWords advertisers and Google. I need to ponder this a bit more. I'll post my analysis and thoughts tomorrow.

[searchengineman]

You need to replicate your results with another AD, before you jump to conclusions.

That must be one hell of a headline. To not be mauled by the "Quality Score Google Bear".

Sad thing is if any Google Bear Keepers, are watching this thread. Don't be surprise if your trick becomes another SPAM sandwich, when they close the hole.


<snip>
Courtesy of <snip>

SearchEngineMan.

[abbottsys]

CONCLUSIONS

Before I give my conclusions let me just summarize this test so folks don't need to scroll through the whole thread.

I ran 2 ads against each other. The first ad (the "real" ad) was one of my long term top performers with all ad copy carefully tuned:

headline
line 2
line 3
display url

The second ad (the "bogus" ad) ignored lines 2 and 3 by simply replicating the headline:

headline
headline
headline
display url

And the test result was that the bogus ad performed just as well as the real ad in terms of clicks, CTR and conversions. The ads are lead generators for a high ticket elective medical procedure (not plastic surgery.)

So, my conclusions:

1) Clearly all I can say at the moment is that this worked for me, and for this particular ad. It would be very interesting to see if others can replicate these results.

2) Notice that the bogus ad is not really bogus, since it uses a powerful and proven headline.

3) Still, if these results can be replicated more broadly they mean that the real power of AdWords text ads lies in the headline, and lines 2 and 3 are far less important, or possibly not needed at all.

4) If it turns out that 3) is true, then both advertisers and Google could benefit if Google displayed reduced text ads on Google.com and the Search Network. A "reduced ad" would simply be 2 lines, the headline and the display URL. How would everyone benefit from this? Easy, twice the number of ads would fit on the SRP, so many more advertisers would get first page exposure. And twice the number of ads would also mean more revenue for Google.

5) It's interesting to note that "reduced ads" are a common format for AdSense publishers who use them on the Content Network. So, Google must have performance metrics for these ads, at least on the Content Network.

6) I would be very interested to hear from anyone who does a similar test.

I do *not* regard this as a "trick", I regard it as a test that could justify running reduced (2-line) ads on google.com and the Search Network. At the very least Google could try this on a controlled basis to gather stats. If successful it could open up vast new distribution inventory for Google.

AWR - is this worth a mention in your Thursday dispatch to the team?

[Mel66]

Fascinating stuff, abbottsys. I'm wondering which of our clients would be willing to have a go at this type of test.....

[searchengineman]

I've been vetoed for this strategy, by our office.

It looks Spammy. Unless you're marketing Parrots or Politicians!

SearchEngineMan

[danielanaidu]

If ad formats are changed to just two lines (a headline and the URL) aren't we going to end up with several ads that look identical? Already right now, many ads have the same headline and are only differentiated by the 2nd and 3rd lines. What will happen if the 2nd and 3rd lines are eliminated?

Aren't we always hearing complaints that the search engines don't give us enough room in the ads to truly specify what visitors will see on our landing pages?

[abbottsys]

If ad formats are changed to just two lines (a headline and the URL) aren't we going to end up with several ads that look identical? Already right now, many ads have the same headline and are only differentiated by the 2nd and 3rd lines. What will happen if the 2nd and 3rd lines are eliminated?

Aren't we always hearing complaints that the search engines don't give us enough room in the ads to truly specify what visitors will see on our landing pages?
Very valid points. But, if visitors are not reading lines 2 and 3 then what's the difference. As regards ads all looking the same, yes, for those folks who just rely on dynamic keyword insertion and do not bother to craft a real benefits oriented headline.

My test, if anything, says you should *really* work on a custom headline. And don't forget, 2-line ads are by no means new, they are used all over the Content Network (actually, these are 1-line ads, the display URL is not used). And these 1-line ads work. I know because I have an AdSense account and test them.

[abbottsys]

UPDATE

Since I gave the results of this test (see the post below, "CONCLUSIONS") the bogus ad has now gone on to outperform the real ad. It is now significantly better. This turned out to be a very strange test indeed. What does it say about adwords ad writing ;)

[abbottsys]

UPDATE

Since I gave the results of this test (see the post below, "CONCLUSIONS") the bogus ad has now gone on to outperform the real ad. It is now significantly better. This turned out to be a very strange test indeed. What does it say about adwords ad writing ;)

FINAL UPDATE
Today I noticed that the bogus ad is outperforming the real ad by 2 to 1. In terms of clicks, CTR and conversions. So the test is over and I'm placing the bogus ad into production. Except, I think I should stop calling it the "bogus" ad, and now call it the "best" ad :)

[Mel66]

Amazing. This is a great example of why everything should be tested - sometimes what we think is the very worst ad / landing page / offer / whatever will perform the exact opposite of what we expect.

[Pakinsto]

Hi abot, i´m gona check your idea, i´ll tell u how it goes

[abbottsys]

Hi abot, i´m gona check your idea, i´ll tell u how it goes
Sounds good. Let us know. To do the test just run an ad like this:

headline
headline
headline
display url

The point is that lines 2 and 3 are ignored and simply replaced by the headline. But remember to use a strong, proven headline.

[Pakinsto]

I belive on u guys, i´m using my best keyword for 72 hours of experiment, at Monday 11th i´ll post my results. I usually have up to 200 clicks daily for that word, so in 3 days, we might have some real numbers to check.

[SearchEnginePPC]

Very, very interesting. I have a few questions for you, abbottsys:

1. Does the 'best' ad use a call to action in the headline? If not, did the previous ad contain a call to action in either line 2 or 3?

2. Is the headline/3 lines also prominently present on your landing page?

3. Does your headline/3 lines for your best ad contain your top keyword? And if so, are other competitors using a similar headline?

Just trying to get a sense why this worked so well, and if it truly is something I could replicate for my clients.

[abbottsys]

Very, very interesting. I have a few questions for you, abbottsys:

1. Does the 'best' ad use a call to action in the headline? If not, did the previous ad contain a call to action in either line 2 or 3?

2. Is the headline/3 lines also prominently present on your landing page?

3. Does your headline/3 lines for your best ad contain your top keyword? And if so, are other competitors using a similar headline?

Just trying to get a sense why this worked so well, and if it truly is something I could replicate for my clients.

Why don't I just give the actual ad. We sell a very high ticket medical procedure called IVF (infertility treatment). It's often not covered by insurance, and so is very expensive, making money a big issue. The "bogus" ad reads:

Can't Afford IVF?
Can't Afford IVF?
Can't Afford IVF?
display url

Keyword is IVF. This "bogus" ad is in rotation with our regular ad. And, as this thread explains, the bogus ad is performing *better* than our regular ad that uses strong sales copy for lines 2 and 3. It's better in terms of clicks, CTR and conversions. The question is why. It appears that, at least for this case, copy lines 2 and 3 really don't matter. Are folks not reading them? The headline seems to totally determine ad performance.

I really don't know if this result can be replicated. I guess you would have to test your particular situation. However, I do find the results very interesting.

[Mel66]

This is brilliant. It makes total sense given the situation. It makes *me* want to go click on the landing page b/c I'm dying to hear the answer!

I think this would work for any similar product or service that's an alternative to an expensive / complicated product or service. It probably wouldn't work for all advertisers, but would be worth testing for the right situation.

[Pakinsto]

Hi there, aprox 72h later, the results from my own "Very Strange Test" by abby ;).

The point was to repeat the same exact keyword in the three line of your Adwords ads. It was my best keyword, in a group of 4 ads

I puted to the test these two type of Ads:

1)

Widget
Widget
Widget
www.ExamPle.com
2)
Widget Local
Widget Local
Widget Local
www.ExamPle.com

For "Widget": the last, 4th, not for much though, - 0.11% from the third and very little conversion rate

For "Widget Local": the first, 1th, with a high CTR, normal being local, and very hight conversion rate.

Conclusions:

The ad "Widget" by itself, didn´t bring much joy to my stats, but it stands for much more that i thought it would. But the word, "Widget Local" it has been very succesuful, so i´ll keep it.

I´ll post more results by the end of the week to recheck.

Curious data indeed.

[abbottsys]

...Curious data indeed.
Sounds like you've had some success with my strange test ;)
I've decided to push this test even further, and test this ad:

headline
-
-
display url

In other words, instead of using the headline for lines 2 and 3 I'm using nothing for those lines. The ad contains headline and display URL only.

[Pakinsto]

Well...as far as i know, it´s ethical, and new, therefore it´s good for advertisement, because that difference might catch the eye of a regular user, used to see lot of text in there...if there´s none...

i´ll wait till u show us some results, because i feel uneasy about playing that way with my best keyword, even though it´s been positive.

And what about check all the possible chances ?

1)

-
Widget
-
www.DisPlayURL.com (example.com)

2)

-
-
Widget
www.DisPlayURL.com (example.com)

3)

Widget
Local
-
www.DisPlayURL.com (example.com)

4)

-
Widget
Local
www.DisPlayURL.com (example.com)

5)

Widget
-
Local
www.DisPlayURL.com (example.com)


6)

----------
-------------------
-------------------
www.DisPlayURL.com (example.com)

6)Nothing at all ?

-
-
-
www.DisPlayURL.com (example.com)

From this point on, just stats can guides us, because if i were using my common sense i wouldn´t do it :)

[abbottsys]

LATEST UPDATE
Well, what can I say? All I can do is give you the results of this truly bizzare test. In a nutshell - I ran 3 ads against each other, and they all performed equally great in terms of clicks, in terms of CTR, and in terms of conversions.

This sounds innocent enough until you see the 3 ads...

With "headline" being my strongest, proven headline, the 3 *equally great* ads were:

Ad #1
headline
line 2 copy
line 3 copy
display url

Ad #2
headline
headline
headline
display url

Ad #3
headline
-
-
display url

The first surprise was #2, and then #3 was a shocker. Do we actually need ad copy ;)

[elf7013]

This is very interesting indeed.
My sense tells me that ad3

headline
-
-
URL

is new, interesting, and outstanding comparing to the competition

If i had a go i would ran a test like:
headline
line 1
line 2
URL

where line 1 and 2 are fill (placement) text as you see for example in creative ads when pics already are finished but you do not have any content yet.

If one of you guys runs this also, maybe this confirms the importance of the headline.

[NewKidOnTheBlock]

Wow, this is really interesting. I can't wait to hear about results from other niches if anyone else does another test on this.

When I read your initial post the first thing that came to my mind was that maybe you shouldn't put the headline there three times, but you should use the headline and then let somebody with no prior adwords experience (or more than just one person..no idea if you have interns or coworkers who have no prior adwords experience) write another ad copy.

That way you should have been able to find out if your ad copy is truly better (and if bothering to learn how to write adwords ads copy really pays off). You could have let them write a headline, too and then try their ads with their headline and with your headline, too.

However, that might not be necessary anymore now that you've tried to just use the headline and nothing else.

I guess it's all about standing out and if you're using a headline and nothing else or if you're using the headline three times your ad just looks different than that of the competition and thus stand out.

Maybe you just started a new trend and people will start to forget about adwords ad copy except for the headline and we'll have to wait a few years until everybody's doing it and an adwords ad *with* ad copy is remarkable again hehe ;)

[abbottsys]

....However, that might not be necessary anymore now that you've tried to just use the headline and nothing else.....
Yes, agreed, there's no doubt this is part of the reason for the success of ad#3. But I'm also getting great conversions, which means the ad is standing out *and* working. I think the bottom line is simply that a headline is *way more* important than many people realize. And a good headline can totally determine ad performance regardless of the ad copy.

Ad#3 is so good that I've now placed it in production and removed my regular ad. Your point about "everyone" doing this is a good one. But, it's not as easy as it seems because it relies on a *really strong* headline, and most ads have weak headlines, especially those that use dynamic keyword insertion.

[NewKidOnTheBlock]

Yes, agreed, there's no doubt this is part of the reason for the success of ad#3. But I'm also getting great conversions, which means the ad is standing out *and* working. I think the bottom line is simply that a headline is *way more* important than many people realize. And a good headline can totally determine ad performance regardless of the ad copy.


I've read two books about copywriting and the #1 rule among copywriters seems to be "take 50% of your time to come up with an incredible headline" (ok 50% is a bit exaggerated, but you get the drift). Very interesting to see that this is just as true for adwords as it has been for direct mail.

I'm only reading about PPC on the side (my focus being more on SEO at the moment), but if I write my first PPC ad I hope I'll remember this and try extra hard to come up with an incredible headline.

How much of your time have you guys allocated to writing the headline as opposed to writing the general ad copy so far? Has it been sort of like 50% or less?

Maybe another conclusion one could draw from this is that (even if we don't all start writing ads that consist of nothing but the headline in every single field) one should focus a lot more of their ad copy writing/testing time on writing and testing the headline than most might be doing at the moment?

And on creating a lot more headlines than most marketers are used to doing so far: As somebody stated above this was also a great example of why everything should be tested even if it doesn't seem 'logical' (as human behavior is kind of unpredictable at times).

[abbottsys]

..... How much of your time have you guys allocated to writing the headline as opposed to writing the general ad copy so far? Has it been sort of like 50% or less?.....
I bet 50% or much less. Many folks even let DKI (dynamic keyword insertion) take care of the headline. My test shows that's a big mistake. Personally, I take a lot of time to write a headline. It totally rules the ad.

[NewKidOnTheBlock]

I've read some stuff about copywriting before and how important the headline is (which made sense to me) but I think without this little test I probably would have still underestimated it (whether for PPC purposes or 'normal' online copy). Thanks for sharing :-)

[shilly]

Google/Yahoo testing, this is fine, HOWEVER if this goes live for all, I feel we should have a choice to opt in/out.
Like content network, a radio button that simplies allows us to decide if we want in or not and allows us to test if our verticle does well or not.