Did Google Just Target Directories

[DaveN]

I have been collecting quite a bit of data just recently and noticed that some WH sites got canned the only thing I can find is a large % of their IBL's are from directories.. is this the end of the road for smaller directories ..

DaveN

[Mikkel deMib Svendsen]

I haven't noticed it (yet) myself but it does make sense - and I trust your data, Dave. It would make sense to lower the value of links if they are all identified as paid, in some way. Directories is a perfect place to start. If they only account for a small percentage maybe they still count but this one is definately one to watch out for ...

[Marcia]

There was a thread started here on June 15th about the Directories category at ODP no longer having any PR

http://forums.searchenginewatch.com/...ead.php?t=6292

It's still PR0. Not only that, but out of all those new "directories" springing up almost daily, most would not qualify for inclusion at ODP, they don't even meet the most basic inclusion standards.

is this the end of the road for smaller directories ..

It may mark the beginning of the end of the road for the business model that's the underlying reason why a lot of those directories are being set up in the first place. That could turn out to be a short-term crash and burn business strategy - it's starting to look that way, isn't it?

[PhilC]

I'd find it difficult to believe. For one thing, Google would need to be able to programmatically sort the directory wheat from the chaf. There are many perfectly good useable directories, and their links are certainly within the realms of natural links, imo.

It would take the examination of a very large number of 'hit' sites to draw any reasonable conclusions as to what caused the hits. I'm reminded of the first widely believed theory following Florida. It was put forward by someone who had examined well into 3 figures of sites - both affected and unaffected - and he came to the conclusion the the Florida update was the introduction of an "over optimisation penalty". The theory was wrong, but it was widely believed at the time and the idea of an OOP got so firmly entrenched that some people still believe it exists. I think the next biggest theory that was put forward was mine. Fortunately, it didn't get so well entrenched because it was also wrong

I'm sure that sites with plenty of IBLs from directories were hit, but so were sites with a negligable number of IBLs from directories - and hit so badly that the devaluing of a very few IBLs couldn't have caused the effect.

[Gerardism]

Hey DaveN,

From your search findings so far, have you noticed any differences from the IBLs from directories,. if they are paid/free to be listed, and if they are paid, do they take any site that will pay or only quality submissions.

If a directory will take any site for a fee without consideration of quality, then I hope the SE's will devalue it.

[Marcia]

Here

http://forums.searchenginewatch.com/...ead.php?t=6002

Notice the part of the quote where it mentions common link sets.

[clasione]

What do you think would be the best way, or most effective way for the major search engines to identify which sites or directories should be targeted if indeed they are going to try and accomplish such a difficult task....

What about sites that have a directory like structure, but are not a directory?

I know that some or most garbage directories are built on similar or even free software which creates thousands of pages on the fly, but some directories are more complex, static, or intergrated with entrie content rich portals....

How would they best target the smaller garbage like directories but avoid a richer site?

[DaveN]

I had Dinner with some Google search engineers ( they even paid .. thanks guys)

It's a funny situation to have a BH and the people that hunt and kill BH sites ... Most of the stuff we talk about is just war stories , what they killed of mine etc..

But the one thing that hit home to me was how Google wanted quality sites and relevant serps... go figure not rocket science there..

so ask yourself do directories add value to the user ??? and what I mean if I'm looking for service I would rather see 10 sites about that service than 10 directories about the service... How many people actually use dmoz to find a website ! or do you just go to Google or yahoo to search there, My point was that most directories are just big scaper sites or will be big scaper sites in the future.

Once up on a time Google had One man spam fighting, they have a lot more these days, and the best way to spam Google is Links... and what do directories provide ... links so where would you put a Google spamhunter..

DaveN

[Mikkel deMib Svendsen]

> My point was that most directories are just big scaper sites or will be big scaper sites in the future

The fact is that a VERY large percentage of the web is really just "scraper sites" - some better quality than others, admitted, but nevertheless just one form or another of reused content. Just look at the news-sites - half of them just write off news from other sources and even the big newspaper fill up most of the pages with pre-written stuff from the major news-wires such as Reuters.

Having said that, it does make sense for engines to wipe out the most crappy scraper sites - including all the new (and I am sorry, but worthless) directories. As I search users I will not miss them

[Marcia]

the best way to spam Google is Links... and what do directories provide ... links so where would you put a Google spamhunter..

So then, Dave - how about for starters they could be putting their spamhunters out and around reading at the SEO forums where all the directories are being publicized to attract customers? They could see patterns in posts and search out the names of posters with + directories and find an awful lot more that way, couldn't they?

Then step two, they could connect the dots between the ones that link amongst themselves and are involved with selling text links - which IMHO is why a lot of those are being set up, as part of linking networks with the final product being selling links - or services based on rankings gotten through multiple linking networks.

I understand the engines have some very nifty tools inhouse that kind of simplify tracking and mapping out linking relationships.

Doesn't that sound logical as a starting point?

[PhilC]

I'm sorry to be a disagreeing voice again, but unless there is real evidence that Google has devalued links from directories, or the directories themselves, I just can't buy it that directories or their links were devalued to the extent that sites plummeted in the rankings.

We are used to seeing loads of people introducing their new directories in our forums, and asking people to submit to them, and it's easy to get a biased view of small directories. But there are loads of genuine directories out there that provide very useful services - niche directories in particular - and links from them cannot be reasonably thought of being different to links from other sites.

I have no doubt that plenty of sites with a lot of directory links suffered in the latest update, but so did plenty of sites without all those directory links. Links from directories isn't a common denominator.

All those new little directories that ask us to submit to them never contain more than a very small number of listings, so they aren't worth the effort to programme against. Scraper sites are mushrooming, and they are worth programming against, but they are not the topic of this thread. Besides, scraper sites mostly contain se results, don't they? And they aren't really disguised as directories, are they? Or is it that I simply haven't come across directory-looking scraper sites?

[Marcia]

You know, if we look back at the first post, what Dave was saying is not related to the directories themselves, but that sites with a large % of their links from directories took a hit.

What are the factors that contribute to ranking boosts coming from links? Is it PR? Is it only a matter of numbers? Does it have anything at all to do with the topics or on-page factors of the sites/pages that the links come from?

I think the engines probably have more serious challenges than the slew of little directories out there - but the issue is whether those types of links, especially if they comprise the majority or IBLs to newer sites that aren't established, in great numbers and accrued quickly, really have any value as far as algorithmic link analysis is concerned. Do they represent genuine links that reflect relevancy or importance, as would a link from a page from a related, on-topic site?

Haven't people been saying for quite a while that the best thing is to get ON TOPIC LINKS? How does that match up to the bill of goods that all you need is 75 IBLs from paid directory listings - and that's how to rank well? It just doesn't compute, and not only that, it's a healthy slap in the face for IR people, and devaluing their credibility..

Maybe it's the criteria for assessing the value of IBLs that could be coming under scrutiny and changing, and that just buying your way to link-pop in order to rank well isn't quite the magic bullet that some claim or hope it is.

[PhilC]

Perhaps the initial post wasn't related to directories themselves, although I think it was. The Subject of the thread is "Did Google just target Directories" - "just" could mean "very recently", or is could mean "only", but the topic is about Google targeting directories in one way or another.

It may be that they are identifying, or trying to identify, different 'types' of links, and factoring the comparitive quantities of 'same type' IBLs into the calculations. But it's only a maybe, and there has always been a plethora of maybes through the years that never turned out to be facts. I've no doubt that, since Google relies quite heavily on link texts for rankings, they continually try to improve the way they evaluate them. It's just the singling out of directories that I can't accept without some real evidence. The reason I can't accept it is (a) because I need real evidence before I can accept a theory, and (b) because I know one site that definitely does not fall into the directory IBLs category, and yet it was hit for six in the update (for people who don't know cricket, that's the same as hitting the ball out of the park - a very big hit).

Incidentally, one thing that can be said FOR directory links is that they are very much on-topic

[DaveN]

PhilC I think we have one of the best link analysis tools in the business, I'm saying that to promote it .. it's not for sale or even going to be used by anyone else..

we have collected very large amounts of data, on IBLs we have built large heuristics of what is a good link and what are bad links... so much so I can list list 10 large corps are using hidden text lol... which sites imo should be banned in the se's etc etc... from this data we saw a large number of sites drop and the binding factor was directory style IBL's.... so that's why I asked the question

DaveN

[PhilC]

That's as may be, Dave, but if links from directories was the binding factor for the 'hit' sites that you examined, then I think you may need to examine more hit sites, because I have one site that was hit badly which has a negligable number of IBLs from directories - so few that devaluing or removing them couldn't have caused the plummet. I'm talking about a mere handful, and DMOZ and Google aren't among them. It only takes one wrong result to disprove a theory.

Did your analysis show hit sites that didn't have many directory links as compared to other IBLs? How many sites were in the analysis? Did any of them escape being hit even though they have a comparitively large number of directory links? Were the hit sites really hit badly or did some of them just drop a little in the rankings? Detailed answers to questions like these would be very helpful for this discussion.

I'm not saying you are outright wrong. After all, Google did introduce 3½ updates this time, which may have changed a number of different things, or they may have changed one thing in several stages. But I am suggesting that you may have drawn the wrong conclusion about what caused the sites to be hit, in the same way that the wrong conclusion was drawn right after Florida, and after the person had examined well over 100 sites.

[Mikkel deMib Svendsen]

> It only takes one wrong result to disprove a theory.

I don't agree to that at all. Most search algo is dynamic in nature so you can usually always find one set of data that "proof" your theory and another set that "proof" it's not true. That dosn't proof anything.

The question here is if you believe Daves data or not. You are free not to - I do. You can't expect Dave or anyone else to provide every single detail about how he collects data and analyze it. It comes down to trust - like in most other discussions on this topic

[DaveN]

my sample data is 250,000 urls if that helps. but we have the ability to go to just under 1 million but the number crunching takes way too long...

DaveN

[Gurtie]

is this the end of the road for smaller directories ..

does that mean you crunched it down enough to say it's the smaller directories rather than just 'directories'? and if so do you fancy defining 'smaller' for us?

Or could it be 'newer', 'lower PR'd' or 'heavily interlinked' directories?

I trust your analysis but I'm thinking I should narrow it down a bit before I e-mail dmoz and demand all sites removed.

[PhilC]

Originally Posted by Mikkel deMib Svendsen

Most search algo is dynamic in nature so you can usually always find one set of data that "proof" your theory and another set that "proof" it's not true.

No so. You can never prove a theory - you can only disprove one, and a single 'wrong' result is enough to disprove a whole theory.

Originally Posted by Mikkel deMib Svendsen

The question here is if you believe Daves data or not.

The problem is that we haven't had any data of any kind - not even an overview. It's not a question of trust. It's a question of somebody saying that my analysis seems to indicate something specific, but what analysis? Is it a secret? We need to know something about it before we can seriously consider the possible conclusions.

It's not a question of trusting the individual. I don't know Dave, but I've no reason to distrust him. But for anyone to seriously think that Google might have targeted directories in some way, it's necessary to know what the analysis consisted of, and some results from it.

Originally Posted by DaveN

my sample data is 250,000 urls if that helps

It does, but only slightly. I'd really like the answers to the other questions as well. If you analysed 250,000 urls, did you know their rankings before and after the update? Do know which of them were hit? Are they sample urls from the top of the rankings and the analysis is in the form of 'I don't see any that have a lot of directory IBLs'? Did you find any highly ranked pages that have a lot of directory IBLs? Did you analyse whole sites, or were they just urls from the serps? There are so many questions that need answers.

This idea is something new that, if true, will impact the way we operate. Therefore it is necessary to know what the analysis consisted of and the basic results from it. A general overview of the analysis with the general results that led you to the directory possibility will do.

[Mikkel deMib Svendsen]

No so. You can never prove a theory - you can only disprove one, and a single 'wrong' result is enough to disprove a whole theory.

I don't know where you get that idea from - that has never been the case in SEO. Often you can point out specific caracteristics with a large group of sites that most likely hit them - and resulted in lower rankings, penalizations or bans. Just because some sites, that share similar carachteristics, did not get hit is not any prrof that the theory and reason for the others sites dropping out is wrong. It just proofs that the engines are not perfect.

In fact, just about any "SEO theory" I can think of can be "proven" wrong with examples that dosen't match. And actually, this is not uncommon for science either. As an example I was actually surprised to learn some years ago that the accepted placebo effect on commercially marketed medication is up to 50% So, even if you have 250,000 different URLs from Daves and they dosn't match Dave's numbers Dave could still be right - even if you have twice that. Algoritmic penalties never hit all - at least not at the same time.