Google's First Click Free

[vicyankees]

Is anyone here familiar with Google's new program "First Click Free"? It's related to their sitemaps feature and tackles the idea of cloaking to allow the search engines to spider content that is behind a registration wall. I am curious to know if anyone is currently participating in this and if you have seen any decline as a result.

[rustybrick]

Are you saying Google has an ad that says "Click Here Free" in it? If so, can you show us?

Is this the first time? I do not know. Could be...

[vicyankees]

I should have explained it better. The issue is for sites that require registration to view their content but cloak the pages to allow the spiders to see the content. The First Click Free Program requires the site to allow the user to view the entire article and should they progress through the site, then they can be prompted to register.

[rustybrick]

As far as I know. Google doesn't like it when you cloak pages.

So you send them a list of pages through sitemaps. The spider then needs to crawl it. If you deliver the pages to the engines but require the users to login, I think that is a form of cloaking.

Sites do get away with this all the time. But doesn't mean Google likes it.

How is this any different then normal cloaking? Instead of getting to the pages through links, it uses the Google Sitemaps feed....

Right?

[vicyankees]

But now Google is requiring you to take down the registration... I have pasted the overview from Google below:

If you offer subscription-based access to your website content, or if users must register to access your content, then search engines cannot access some of your site's most relevant, valuable content.

Implementing Google's First Click Free (FCF) for your content allows you to include your premium content in Google's search index. First Click Free has two main goals:

1. Including highly relevant, premium content to Google's search index provides a better experience for Google users who may not have known that content existed.

2. Promoting sales of or subscriptions to premium content for Google partners.

To implement FCF, you need to allow all users who find your page using Google search to see the full text of the document that the user found in Google's search results, even if they have not registered or subscribed to see that content. Thus, the user's first click to your premium content area is free. However, you can block the user with a login or payment request when he tries to click away from that page to another section of your premium content site.

Thus, FCF is designed to protect your content while allowing for its inclusion in Google's search index

[Jeff Martin]

Cloaking is probably the wrong term to use here. If cloaking is being used then I would recommend a change.

You could use a list of SE IPs and allow them into your site (use the 'no cache' tag) and deny all other IPs. You could use Ralph's regulary updated lists over at Fantomaster.

[dannysullivan]

vicyankees, I'm assuming you've heard from someone beta testing this program or perhaps have been offered it directly by Google, correct?

News of a Google "Premium Content" Program Begins Surfacing
http://blog.searchenginewatch.com/blog/050630-180813

That's on our blog yesterday, covering the information about the program that's emerged.

Google Indexing Subscription Content
http://www.betanews.com/article/Goog...Web/1120164520

It the Beta News article we blogged, that broke the story.

My understanding is no cloaking would be required, under the program. IE -- you'd be doing this with Google's permission, and if you detected it was someone coming from a Google referrer, you'd let them in.

[rustybrick]

Oh the "deep web" that Yahoo announced and now Google is following suit with "Premium Content". Ohhh. Thanks Danny for clarifying.

[dannysullivan]

I'll be very curious to see how this works in practice, as well. It seems pretty easy to abuse.

The idea is that you have premium content. Someone clicks on something they found in search results, you should them that as a "freebie." Then after that, they have to pay.

Well, if they only wanted that article, then you are unlikely to gain. More to the point, it seems pretty easy to fake a Google referrer to do this over and over again. Or I just kill my cookie, then search for other things I might want, click again.

I understand why they want to do First Click Free. They want a compromise solution so users are dissatisfied by not getting to what they actually wanted. But it will be interesting to see if publishers find this that worthwhile. I'd have to debate it heavily with our own premium content. I'd love to have that indexed, but I'd be afraid the First Click Free would just devalue being a member.

[vicyankees]

From our standpoint, we look it at as losing 100,000's of pages in the index. We do not charge to see the content, but merely ask for a simple registration.

[dannysullivan]

Yes, big difference if you aren't changing!

[vicyankees]

I was unaware of Google "Search Experience" department. But it seems to me that this is going to become the new way to combat cloaking. Their implementation is cloaking in its simplest form. We've already seen a significant drop in index count over the past 2 weeks, partially attributed to the Bourbon update - but to lose 300,000 pages from the index is extraordinary. Their premium content that you mention previously is an entirely different item with different specifications although similar. Have you heard of anyone else speaking of "First Click Free" and any implications that they have seen as a result?

[esoos]

I'd have to debate it heavily with our own premium content. I'd love to have that indexed, but I'd be afraid the First Click Free would just devalue being a member.

I'm in the same boat. We've got about 600 SEM articles and reports that are effectively invisible to search engines. Getting that content indexed would be nice, but our subscribers pay for access to privileged content.

With First-Click-Free, anyone with any Google-hacking skills would easily be able to access all that content freely, and there'd be no value in a membership subscription.

I'm sure most subscription sites will see it this way. Forrester research is selling reports for US$200 to $300 and up. I can't them being interested in that content being freely available.

The executive summary or abstract is probably the way this is going to go, the way you see with many sites in Yahoo Subscription. That's the best way to maintain the exclusivity of the content while still giving people a taste.

[Mikkel deMib Svendsen]

Yes, this is definately way to easy to abuse. Faking HTTP referer is the easiest thing in the world and will leave site owners with no real protection. I am actually surprised to see Google come up with such an idea - I mean, they may be gready, evil and all that but they are certainly NOT technically neewbies. You can do better than that, Google ....

[Jeff Martin]

vicyankees, if you need help setting up an IP based system to allow the SEs in and keep everyone else out who isnt on a subscription I can help you with the general details.

Of course, any serious cloaker worth their salt could help you to...

[Chris_D]

The issue is for sites that require registration to view their content but cloak the pages to allow the spiders to see the content.

We do not charge to see the content, but merely ask for a simple registration.

So isn't the simple answer to 'preregister' the spider so it doesn't have to register by itself (because it can't)? Thats been a 'workaround' for years...

[Mikkel deMib Svendsen]

So isn't the simple answer to 'preregister' the spider so it doesn't have to register by itself (because it can't)? Thats been a 'workaround' for years...

Now THAT is funny!
I have never seen the term "preregister spiders" used to defend cloaking - and that is really what it is. As most of you know I have no problem with cloaking but please let's stay with the fine term we have: Cloaking - changing the term to "preregister spiders" will NOT change how engines look at the results of it.

I have a bunch of sites that people claim I am cloaking. No, no, no, I just "preregistered" the spiders - and oh, yes, users can't register

[Chris_D]

Mikkel,

If I have 'registered' with e.g. a news source - where no payment was required - I get to read the content. If the same news source 'pre registers' the spider - then the spider gets the same identical content.

Maybe you'd better define 'cloaking'.

Google says

Don't deceive your users or present different content to search engines than you display to users, which is commonly referred to as "cloaking."

How can it be cloaking if you display the same identical content to the SE spider as you display to the registered user??

Maybe you haven't been cloaking after all Mikkel?

[Mikkel deMib Svendsen]

I think Google describes it just fine. I am not registered, so if you pre-register Google you WILL show them something different from what I get. And, that is exactly the core of cloaking. Admitted, i CAN get acess to it if I register, but I have just never seen anyone claim that payment or not is what seperates cloaking from not. It dosn't make sense to me. Weather I have to pay or not to get registered is not the issue - the issue is what you show me (as a not registered users - as I naturally will be to start out with) and what you show Google.

[littleman]

Not to get caught up in the minutia of definitions, but technically this is cloaking. It is a two step process first one allows Googlebot in via IP Delivery (cloaking) then one allows the surfer from Google in via HTTP_REFERER (referer based cloaking). That's really a side topic though.


What I am more interested in is what related services Google will roll out next?

My bet? They will work in a subscription solution within a year.