Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engines & Directories > Google > Google Web Search
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 11-12-2004   #1
alyt
Newbie
 
Join Date: Nov 2004
Posts: 3
alyt is on a distinguished road
Question Locked content on Google

Hi

Have searched this site for a while but can find no answers to help me resolve something.

I run a site of subscriber-only content and I want to submit all that content onto google.

How can I allow google access to the content for indexing without compromising subscriber-only access? And how can I do with this without cloaking or redirects?

Cheers

Al
alyt is offline   Reply With Quote
Old 11-12-2004   #2
Nick W
Member
 
Join Date: Jun 2004
Posts: 593
Nick W is a jewel in the roughNick W is a jewel in the roughNick W is a jewel in the roughNick W is a jewel in the rough
You cant.

What many people do though is to allow the bot AND the user to see the first 200-300 words, it's enough to get indexed, works nicely as a teaser, doesnt involve cloaking or redirects and keeps everybody happy
Nick W is offline   Reply With Quote
Old 11-12-2004   #3
Noustie
Member
 
Join Date: Sep 2004
Posts: 7
Noustie is on a distinguished road
Our Solution...

My company has encountered a similar problem, though not exactly. Basically we have content that we want the bots to reach, but we don't want to provide a list of that content in the form of links for reasons involving our competition. What we've done is to link to the content using clear pixels on our site, so that your average user won't see it or ever click there but the bots can still spider it. Our tactic is to do one clear pixel in the footer that links to a page of clear pixel links for the content. No one but the bots have gotten to it yet, and we haven't seen any penalties.
Noustie is offline   Reply With Quote
Old 11-12-2004   #4
mcanerin
 
mcanerin's Avatar
 
Join Date: Jun 2004
Location: Calgary, Alberta, Canada
Posts: 1,564
mcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond reputemcanerin has a reputation beyond repute
Noustie, that's a VERY dangerous tactic and I advise you to find a different way ASAP.

SE's hate that and will penalize it wherever they find it. It's just a matter of time.

Here is the problem, you guys. Lets say this works and you now have a fully indexed site that shows up for your keywords. Then what?

I type in a query and see a link to a page that might answer me. When I go there I get a "Subscribers Only" screen (or not authorized). I get REALLY annoyed. I report it as a bad or broken link. It gets removed. Or I go start using someone elses search engine that doesn't effectively make me sign up for something different every time I do a search.

The bottom line is that trying to tell google something and trying to avoid telling regular people that same thing is cloaking. By definition. Regardless of the tactic you use.

Take a look at this very website: www.searchenginewatch.com. You will see that they have a bunch of free content, and also a bunch of partial articles that you have to log in to in order to read the rest. This is VERY effective (it's how I became a member a long time ago).

Bottom line, if you are trying to figure out how to show an SE one thing and a person something else, you are heading in the wrong direction. You can show the same thing to both and still encourage signups.

Ian
__________________
International SEO
mcanerin is offline   Reply With Quote
Old 11-12-2004   #5
alyt
Newbie
 
Join Date: Nov 2004
Posts: 3
alyt is on a distinguished road
Smile great points...

Yeah I agree with you mcanerin - problem is that we have a product that is very valuable and we have to charge for it and we cannot give away more than a handful of any articles at any given time.

Not enough to do us justice on any SE....I have no problem letting people know even at the SE results level that it is locked content but I cannot do that as far as I can tell as google ignores meta tags now.

I really think there is an awful lot of content out there that is locked and not being spidered...

Surely google could come up with a solution...

The only other way I have thought of thus far that is effective and legal is to use a script that says any user from the google bot specified IP range is a google bot and the system should recognise it as a subscriber etc.

That way google gets in and spiders...

This started by the way as we were approached by a SEO who would create a mirror of the site for google to index and then redirect and they claimed they could redirect 'legally' and they then charge a click through rate based on the redirects. But whilst we liked the idea of exposing our content to the crawlers as our content markets itself we remained unconvinced of their methods.

any other thoughts...

cheers
alyt is offline   Reply With Quote
Old 11-16-2004   #6
Jeff Martin
 
Jeff Martin's Avatar
 
Join Date: Jun 2004
Location: Dallas, Texas
Posts: 364
Jeff Martin is just really niceJeff Martin is just really niceJeff Martin is just really niceJeff Martin is just really nice
Quote:
Originally Posted by alyt
The only other way I have thought of thus far that is effective and legal is to use a script that says any user from the google bot specified IP range is a google bot and the system should recognise it as a subscriber etc.
And thats really the best way. You can buy a subscription to Ralph's (Fantomas) spider list and match the users IP against it to allow spiders in only then just use a no arcive tag to keep the meat of your content out of the cache.

If you go by user agent only then eventually it will bite you in the rear...
__________________
Jeff Martin - SEW Moderator
Vericlix
Jeff Martin is offline   Reply With Quote
Old 11-16-2004   #7
alyt
Newbie
 
Join Date: Nov 2004
Posts: 3
alyt is on a distinguished road
Coolio...

Will progress down the route and report back...
alyt is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off