AdWords To Begin Crawling Landing Pages & Analyzing For AdRank

[fulton savage]

I read this notice today, but the explanation seems out of place.

Important notice: Please be advised that Google has modified the standard terms and conditions of the AdWords program. As stated in the terms, your continued use of AdWords indicates that you accept these new terms and conditions.

Review the updated terms | Review the previous terms | Learn more | Dismiss this message

On the "Learn more" link I read..

...
We've also added some language to anticipate Google's retrieval of advertiser landing pages. To further improve program quality, our system will soon visit and evaluate all landing pages specified in AdWords ads. The quality information collected will affect AdWords account performance in the future. If a landing page has informative content related to its AdWords ads and keywords, these keywords will receive higher Quality Scores and potentially lower minimum cost-per-click bid (CPC bid) requirements. Poor quality landing pages or those that restrict visits by our system are likely to experience a decrease in quality scores (and a potential increase in CPC bid requirements).

Learn more.
...

I thought this was happening for a few months already?

Also interesting:

Additionally, due to Google's progress in making an AdWords API available, we're asking users not to "screenscrape" AdWords web pages. We believe screenscraping may negatively affect the performance of AdWords and that more efficient results can be obtained using the AdWords API.

[dannysullivan]

I've posted more about this here on the blog: Google AdsBot Now Coming To Assess Your Landing Pages, Will Impact Your AdRank.

Basically, they were getting some of the landing pages through other crawling but now will get all of them through a dedicated spider. You can opt-out -- but if you do, your quality score will suffer.

[Richard Gregory]

Doesn't it feel like the lines between paid and organic search are starting to blur at Google?

[Chris_D]

To further improve program quality, our system will soon visit and evaluate all landing pages specified in AdWords ads

Its a shame Google didn't start at the other 'end' - instead of evaluating all landing pages specified in the adwords ads - Google should start evaluating all their MFA scraper 'partner' sites included in the Google content network.....

Personally - I think Google are sniffing around at the wrong end of the dog.

[fulton savage]

Yeah, but I was actually disappointed to find that landing pages weren't being completely examined until now. The tighter the system gets the better my company will do because I'm on top of things (or at least I thought!).

And Richard, at least a very distinct division between paid ads and natural results remains on the SERPs at Google, whether amateurs know the difference or not. The secrecy behind the pages we will never understand, so we should be grateful that the surface is still easily deciphered.

[Chris Boggs]

Originally Posted by Chris_D

Its a shame Google didn't start at the other 'end' - instead of evaluating all landing pages specified in the adwords ads - Google should start evaluating all their MFA scraper 'partner' sites included in the Google content network.....

Personally - I think Google are sniffing around at the wrong end of the dog.

Amen brother. We were talking about Supplemental Indexed pages in another thread and I wondered if some sort of folksonomy could be used to combat no-longer-relevant pages in that index. I wish there was a way to do the same for scraper results.

[Mikkel deMib Svendsen]

One of the reasons we use PPC with clients is that they want to promote pages that are virtually un-crawlable and with no real text-content, such as pure graphics, Flash, video and other multimedia. If Google crawl these pages they won't find any keywords that match the ads and as I understands it my clients will then be punished for it.

I hope Google realizes what they are doing...
If the prize goes up for my clients with pages like this it will just make it more likely that they chose a cloaked solution instead of PPC - cloak the un-crawlable pages and organically rank them. That works great. The client gets the good traffic, I get the money and Google gets nada! Is that what they want? I don't care much - my job is just to serve the clients with what they want and make sure I profit from it, one way or another.

Another solution, that I need to test, may offcourse be to just to cloak the adbot

[projectphp]

I really like the idea personally. I hate when queries like [accountant job perth] end up at a generic job search page. Encouraging people to send ads to better pages is a great idea.

Mikkel's examples are problematic, VERY problematic, but I am assumming that this will be accounted for, and not strict in the sense that the organic results are. I would really like to see the rules for scoring well on AdRank be VERY transparent, to the point that we know what they look for. To me, that is the goal here, SEs and avdertisers working together in order to provide users witha better experience, and rewarding sites that do it well.

[Mikkel deMib Svendsen]

Thanks, if we are told exactly how the scoring works then my adbot cloak will work perfectly

[Wilksy]

Good call Mikkel, Google are getting plain silly these days.

[projectphp]

I still say its a great idea. YV stations vet their TV ads, why shouldn't SEs make sure that ads are relevant to a page? Personally, I think this will target broadmatch more, with a search for [gift delivered hicksville Namibia] no longer showing those broadmatching on gifts.

[Mikkel deMib Svendsen]

Yes, the goal is fine, the idea is good, but the solution is totally bad.

[Robert_Charlton]

On some sites, because of the directory structure I inherited, I've been using the robots meta tag, not robots.txt, to block spidering of landing pages. The new AdWords Help Center info page doesn't talk about this. Anyone have an idea whether AdsBot will disregard the meta tag too, so I can leave the landing pages where they are.

Otherwise, I have to build a new directory structure for the landing pages, with all sorts of changes with AdWords down the line.

[fulton savage]

Originally Posted by Robert_Charlton

On some sites, because of the directory structure I inherited, I've been using the robots meta tag, not robots.txt, to block spidering of landing pages. The new AdWords Help Center info page doesn't talk about this. Anyone have an idea whether AdsBot will disregard the meta tag too, so I can leave the landing pages where they are.

Otherwise, I have to build a new directory structure for the landing pages, with all sorts of changes with AdWords down the line.

What makes you think the robots obey any of your "blocks" ? This is one question I have for everyone, as I always questioned the meaning of such actions.

[Mikkel deMib Svendsen]

What makes you think the robots obey any of your "blocks" ?

All major search engines and most minor ones fully respect the robots.txt as well as META-robots "standards". You can find a few abscure examples of where they did not, but that is definately unintentionally.

However, those standards only "block" crawling - not indexing.

[Jonathan Mendez]

I really would like to know what Google will do with landing pages that are delivering content dynamically via Ajax. The benefit of Ajax and other RIAs like Flex is to provide relevant and simple user experiences. Ideally that's what Google wants. These dynamic pages should get quality scores in spades. Could they actually be penalized under this bot?

[projectphp]

How can you deliver people to an AJAX page about nothing? Surely, you have something on the page!

If not, and it is pure javascript, then just add some <noscript></noscript> content.

I've been using the robots meta tag, not robots.txt, to block spidering of landing pages

Now there is bloody good question! I hope we can get that answered...

Yes, the goal is fine, the idea is good, but the solution is totally bad.

Lets see how it works first. I agree that PPC is ideal for flash, and one would hope that they account for this. Only time will tell I guess.

[Mikkel deMib Svendsen]

How can you deliver people to an AJAX page about nothing? Surely, you have something on the page!

In an AJAX application you don't neccesarily have anything but the AJAX engine on the page - the rest is retrieved asynchronously of the user interaction. Kind of like a long (endless) Flash movie.

Unless the AdBot actually execute the AJAX application they won't find any content and I very much doubt they will.

Off course you could ad some NOSCRIPT content but given the dynamic nature of AJAX what would you put there? The first part of the "movie" - the middle, the last? In any case, this will be no different than cloaking - only cloaking would make it easier to manage.

[grnidone]

How is a machine going to be able to understand if a landing page is a "quality page" or not?

[Marcia]

How is a machine going to be able to understand if a landing page is a "quality page" or not?

My guess would be that they'd use some technology similar to what they use for figuring out the contextual criteria for Adsense. I imagine there would have to be some semantic clues found on the landing pages that would show that they're on topic for the ads.

Not a bad idea, since a lot of ads that run on Adsense have nothing to do with how the ad reads on the landing page, they're nothing more than a search form or a sign up for a freebie deal - nothng at all related to what they're advertising for or the publisher sites they're running on.