Dear Matt Cutts, I Want To Know About...

[dannysullivan]

Next week, I'm doing a special edition of the Daily SearchCast with Matt Cutts when I'm at the Googleplex. More about when and how to listen is here: Next Week's Special Edition Daily SearchCast With Matt Cutts, Live From The Googleplex (http://blog.searchenginewatch.com/blog/060510-140615). Matt's also got a post about this here: Danny in town next week (http://www.mattcutts.com/blog/danny-in-town-next-week/).

Got things you'd like to see us discuss? Drop your suggestions here.

[GaryTheScubaDiver]

Is all the muck with the DC's completed? I've got clients from UK to Costa Rica flipping out.

[joeblstrn]

sure... talk about pages being dropped, and then appear suddenly. Many of my sites are ranking one day, gone the next and then back again. I am not talking change in postion. I am saying ranking in the SERPS one day and poof gone the next. Then, hey back again.

[sulamith]

That would be my number #1 question to. Why is my site there, prominently listed one day and then the next day, not to be found AT all, in 70 plus pages of returned results?

Secondly I guess I'd like to know why the quality of the results is just garbage now?

[JEC]

Those are good questions. Here's one of my own.

Matt, I'd like to ask you what benefit (if any) you have seen the "sandbox" have on the filtering of sites that are purely setup as spamming sites for link farms and such and removing them from the Google index? And, since there are plenty of sites like these out there that pre-date the "sandbox", how does Google foresee filtering those pre-existing sites from relevant searches in the future? Many of our collective search results still continue to list these types of sites. We all know it's an enormous task, but shy of us reporting each and every spammy site, what is Google doing to naturally filter these sites out, remove them from the index and improve the overall quality of the end user search results?

Thanks,

JEC

[sulamith]

Excellent question! I can't wait to hear good answers!

[sulamith]

Two more questions I've thought of:

1) Why does Google apparently put so much weight on the links to a given site, knowing that black hats use and abuse this to get their sites ranked higher? Given that any blogger can set up multiple webpages with outbound links to their site--why would they place so much weight on links in terms of ranking webpages?

and that leads me to this question, something I alluded to earlier, and which is also mentioned elsewhere in another thread on this board:

2) I'm really disturbed by the quality of the returned results on Google in recent weeks. It seems as if non-relevant pages for a particular search are being ranked far higher than pages that are much more relevant to that particular search term. Here's what I mean: Why is a page which merely mentions a thing or person ONCE showing up ranked far, far ahead of a high-quality website totally devoted to a person or subject? The assigned page rankings just seem very "wrong" to me on a number of different searches I've tried. Has Google lost the forest while looking for the trees? Is Big Daddy really an "improvement" or is it in fact dragging the quality of Google down for the average user?

[richardb]

Hi Danny

Would like to hear the long term approach to take 12 – 24 months, I’m assuming that G does plan long term.

The do’s and don’ts. Stuff like “burns those blogs and include the copy as part of your main sites FAQ’s and news items…” and then I’d like to see DS, Greg, Todd, DaveN et al’s theory of what was said. Hey but that will happen anyway ;)

Thanks

Rich

[Gurtie]

I know that they try and do things algorythmically but if they really want rid of spam why don't they manually ban sites which are so obviously spamming dead blogs in the meantime? there are even google employees abandoned blogs with hundreds of comment spams hanging off them.... (and not all recent and viagra, user experience surely does matter for mobile phones, mortgages ad other stuff 'real users' search on? - some sites which were comment spamming 12 months ago are still in the index)

and actually, why cant the algo deal with that anyway? Just processing power? It doesn't look like they care very much to be honest :rolleyes:

I have another couple but he won't answer them :p

[scrubs]

richardb hit the nail on the head I feel, I am interested in knowing what Googles plan for the future of search is?

There have been many problems and discussions in these forums about the last shake up, but that is part and parcel of our industry and dealing with these changes.

It would be nice to hear what emphasis they are going to put on producing the best results for the end user in the next 12 months? :)

[sootledir]

At the Orlando PubCon in 2004, he spoke about Google and how they dominated search because of four factors. Two of the factors were freshness and comprehensiveness. It appears they're falling behind in both categories. Is this some sort of strategic shift?

[traian]

Is Google going to rely(at least 1%) on the information provided by the users of google Co-op?

[richardb]

OK Danny to bump the thread...

I’d love to hear how G being a better search engine will not affect their revenue in terms of advertising! The bottom line is they are now in the biggest dichotomy of all time…

…how do we keep our market share (in terms of search) and yet still hit effective revenue re. advertising… :confused:

In the UK we call this conflict of interest.

Hi ya Mat, bet this one doesn’t even make the b list!!!


Rich

[Brian M]

There are still a lot of really old supplemental results showing up for pages that have not existed in more than a year. When will these finally be cleared out?

[Nacho]

Back from previous thread (http://forums.searchenginewatch.com/showthread.php?t=6718
), but never got to any conclusions since only search engines hold the answers. We know link popularity follows to the new destination on 301s.

What I would like to learn from a gudeliness point of view, is if Google will consider the URL's age as part of the time factors and it has any value in the algorithms, and will that transfer over to the new URL after doing a 301?

EXAMPLE

In the case it's not the domain that is being 301ed but an actual set of URLs, such as:

domain.com/old-category1-name.html

gets 301 to

domain.com/new-category1-name.html

By this we know it's giving the search engines the instuction to notify that a page was permanently moved to a new destination, as well as link popularity will follow. The question is, will the age recorded or any other time-sensitive factors in the search engine's records for "old-category1-name.html" follow down to the "new-category1-name.html"?

Thanks Danny!

[Chris_D]

Don’t use “&id=” as a parameter in your URLs, as we don’t include these pages in our index.
http://www.google.com/intl/en/webmasters/guidelines.html

I haven’t ever understood under what circumstances pages with urls which include &id= in the URL string are actually excluded from the index.

Clearly - pages with URLS which include the parameter “&id=” in the URL string ARE included in the index. Look at some of the URLs returned from these searches:
http://www.google.com/search?num=100&hl=en&lr=&as_qdr=all&q=allinurl%3A+id+%26+%3D
http://www.google.com/search?num=100&hl=en&lr=&q=site%3Aaddons.mozilla.org+%26id%3D
http://www.google.com/search?num=100&hl=en&lr=&q=mambo+%26id%3D&btnG=Search

Could Matt please clarify under what circumstances pages whose URL string includes “&id=” are excluded from the index? More than two occurances of ‘&id=” in one URL?; more than three occurances?; or some other criteria?

Either way, the guideline as written is currently misleading, as some pages which include “&id=” in their URL are in fact included in the index.

[Spiel]

I would like to know if there is a quicker way to get sites that have been blacklisted back into the search engine. We have 2 clients whose sites did not meet Google's guidelines. We fixed them about 8 months ago. They are still not listed. I got an e-mail from someone at Google stating that the robots will get to them in a couple of months and then they will be listed again. They still are not in there. Why can't Google just have them be listed again rather than depending on the robots to find them?

Is there something I can do to get this to happen?

Thanks!

[kleenex]

What level of on hands work do you have in regards to the “data refreshes” that affect SERPS and when can we expect to see more from the crawl team with regards to discussing it more in-depth to a broader audience? (i.e. like you do with us)

Why is it taking so long to have the constant and consistent indexing issues resolved? Since Big Daddy was unleashed the same types of web sites pop up that never used to be there before. I am referring to large scale shopping sites, major news articles, and irrelevant and sites that are circa 1995. In some Niche’s it really does seem like the little guy is getting pushed out of page one.

I can understand that issues may arise, but to say nothing is wrong is only adding fuel to a growing trend among Webmasters. Why not simply say, we are having an issue with some things? It would come across much better than showing pages that really are the webmasters issue. I think that they are justifiable, but I also think that recently there are more Webmasters that are not spamming or making mistakes on their end than the ones that are.

Thank you.

[Brian M]

Could Matt please clarify under what circumstances pages whose URL string includes “&id=” are excluded from the index? More than two occurances of ‘&id=” in one URL?; more than three occurances?; or some other criteria?

Is there a query string that webmasters can use that will NOT be indexed so we can track both internal and external click traffic without creating duplicate content?

Thanks,

Brian M

[Robert_Charlton]

nofollow: Seems nofollow has evolved on Google from a link attribute for blogs, signaling that the link was to a site we "could not vouch for," to an attribute signaling that the link should not pass PageRank. To me there's a subtle change in this that's not clear....

- Does the use of nofollow confer any stigma on the destination site?
- Would it be appropriate to use nofollow, eg, to allow safe cross-linking of friendly sites, to allow navigation between the sites without giving Google the impression of linking to gain PageRank?
- If "cannot vouch for" is no longer the appropriate language to describe the intent of nofollow, how would Matt currently summarize what the purpose of nofollow is?

Scraper sites: How concerned should we be about seeing our content duped on scraper sites or being recipients of hidden links from these sites? Are we wasting Google's time and our energies reporting or worrying about this stuff?

sandbox: In relation to the non-existent sandbox, are there any guidelines Matt can discuss about making large content additions and/or structural changes on existing sites? No change in subject matter area here... just a site rebuild.

canonical question: Can Google currently sort out the difference between domain.com/ vrs domain.com/index.html, or should we be cleaning this up, perhaps using absolute urls, on sites we work on?

Thanks.

[Relevancy]

1: Does having a sitewide navigation/layout on the entire site going to hurt your site or trigger any duplicate filters? Example is where the nav and site layout is the same and only the body text in the page is different.

2: How does Google plan to better handle click fraud?

[gsitts]

My question would be broader in nature. What is Google doing to better list good clean useful sites that do not have or use external links?

We run one site that has no inbound or outbound links... they are not appropriate to our business model. We use PPC on both Yahoo and Google with very satisfactory conversion ratios.

Our site is tuned for meta tags, internal text, internal links and alt tags to be appropriate for best user/customer experiences. The site is not pimped for Google. As a result none of the pages show up in the first two hundred of organic Google search though they do come up in Yahoo.

I realize that Google has built a very good business out of its PageRank system. But I have this gnawing feeling that PageRank worked initially because it was so new... the blackhat SEOs were not optimizing for page rank, so there was legitimacy to the algorithm. That situation has changed so dramatically it has to be a frequent subject of internal Google discussions.

Our particular site benefits as the quality of the Google organic search results deteriorate. Our PPC ads increasingly stand out in appropriateness. In the short run, Google benefits financially as we are pushed further into PPC. But they are also wandering further from their earlier position as a high quality search site, drawing slowly closer to becoming a site of paid ads and junk copy. That can't be a good business model. At some point they have to take some action to rebalance between paid ads and free listings if they are to retain their dominance of the hearts and minds of search engine users.

[glengara]

Will "similar pages" ever be based on topic rather than linkage?

[randfish]

In the event that a company has multiple portals for multiple countries (i.e. a .com, .co.uk, .com.au and .ca) and is serving both duplicate and unique content across each, what is the best practice for the pages on the multiple domains, assuming the company is unable or unwilling to canonicalize the shared content on one domain (for segmentation purposes)?

[seoqueen]

I would like to know if Google or other search engines for that matter sees www vs non-www as dup content then how one fixes the issue of having www.domain.com and www.domain.com/index.htm included in the results.

I'm technical savvy enough to be dangerous, but not to the extent of figuring out how to have the search engines only index the home page once?

So is this something Google will be looking into to either ignore the dup content on the 2 urls of the home page and only give credit to one or continue to index both and give out penalties?

Also, what does it take to get a new domain out of the sandbox after a year and a half? The site is in good standings with SEO tactics and building quality links...what else does it take??

Thanks,

Seoqueen

[PhilC]

I want to reiterate what a number of people in this thread have said. Tell us what the hell is happening!

So many sites are having pages dropped, put in Supplemental, brought out of Supplemental, pages built up again, dropped again, old caches, new caches, etc. etc. etc. It's becoming very annoying, and it really does matter to people.

Some people think it's because there's a complete index rebuild going on. Tell us if that's the case - most people will be content to wait if they know it's a just a matter of time.

Some people think it's because Google is short of space and is dumping some stuff to make way for other stuff, because of what Eric Schmidt said. Tell us if that's true. At least people will know that it's not their fault, and that all they can do is wait.

Some people think that Google really does have a technical problem, because of what either Larry or Sergey said. Tell us if it's a problem that you are trying to find and fix. At least people will know that you'll fix it sooner or later, and that it's not their fault.

Whatever is happening, it's happening big, so tell us something about it. Keeping silent about it, when you are so helpful so often, is the worst thing that you can do for people who are badly affected. People will understand and generally accept it, if they are told what's happening. There's a lot of goodwill towards Google, but silence in the face of the current fiasco is bound to erode it.

[sem4u]

Good question rand. I want to ask the same thing. Maybe we could have a country tag so that sites would only come up in the correct SERPs, e.g. a UK site comes up on Google UK only. Not perfect but it would help.

[GaryTheScubaDiver]

Danny,

I need to preface my "basic" question I posted just after you initiated this thread. I just got back from a client in Costa Rica and I am about to go to London and meet with another client that I am sure will have the same issues.

So here we go.

Matt,

We understand the issues with Big Daddy and the apparent algorithm problems pulling the spam directories and overloading your servers, and not doing what it was supposed to do. We appreciate Google pausing as well....good anticipation.

I can't agree with reverting back to the Jagger data but if that's all you had to work with...that's all you had to work with.

My concerns are many, but I'll hit the top 3.

1. Duplicate results - Will the Big Daddy update solve the multiple results issues on queries that I've read about as being part of the spam filters that G is trying too solve?
Here is an example - search Costa Rica office rental (http://translate.google.com/translate?hl=en&sl=es&u=http://www.google.co.cr/&prev=/search%3Fq%3Dgoogle%2Bcosta%2Brica%26hl%3Den%26hs% 3DXMK%26lr%3D%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official)

Dual listings in the top 4 results.

Where does this secondary listing come from? Is it a result of the spider looking for the next page based on page navigation?


2. Are you familiar with the "anonymizer" software? I am not sure exactly how it works but the URL string looks like this:
http://www.1costaricalink.com/cgi-bin/nph-split2.pl/010100A/http/www.ocotaldiving.com/specialcourse/index.htm
http://www.google.com/search?hl=en&lr=&q=site%3Awww.1costaricalink.com+.pl

Is this a scraper?

Note that the string has captured the BH's site and attached itself to the page that G is indexing and in essence stealing their traffic.

This is a problem because when you go to this page and click on "contact us" is will redirect you to the BH's contact page. So this guy is stealing traffic from unknowing sites. Then he turns around and sells the leads.

These guys are building massive DB's in attempts to build content for the end-user, and this guy is stripping his potential clients away.

By the way, these particular guys have no recourse because their is no such applicable law to regulate it in their country so they on Google to award their efforts...that is not happening.

3. Will Big Daddy take care of the ridiculously obvious spamming sites such as this:http://www.costarica-beachrentals.com/
or this:http://www.1costaricalink.com/cgi-bin/nph-split.pl/010101A/http/www.hotelarenalsprings.com/ingles/hist2.htm

- look for the hidden text.

I appreciate the fact Matt that Google is trying to deliver genuine and relevant content. I preach this to my clients daily. But the problem with this is that because of these circumstances they want me to concentrate on grey hat techniques.

I try to convince them that Google will eventually reward unique content that actually provides the end-user with exactly what they are looking for. These companies, based on my recommendations, and your advice, have spent 10's of thousands of dollars on hiring people to create unique content that is relevant to their potential clients are seeking.

Your current algorithm is rewarding spam directories and putting the black hatters above the genuine guys out there trying to operate based on Googles stated rules and parameters.

So how long will this take?

My clients spend tens of thousands on great content that provides the end-user with exactly what they are looking for based on their queries, but Google is a constant disappointment.

I understand you have your own issues, but Matt, as SEO's we have to answer for these recommendations...We are doing exactly what you have said and our clients sites are dropping in SERPS from page 1-2 to page 20-30.

So my question (lol) Matt is this, should I go ahead and implement BH now, or later? We are trying to give you what you ask for, but the black/red line only extends so far.

Your are not responsible for these companies, or their dependency on your positioning. We understand that. But we spend alot of money following your recommendation with even worse results month after month.

We, as Webmasters and the like, are the people that spread your name and have made Google what it is. We appreciate the tools, advice, your blog and the feedback, but your slapping us, as well as our clients, in the face with the recent updates.

What if anything can you tell us that we can count on and calm our clients down?

[GaryTheScubaDiver]

In the event that a company has multiple portals for multiple countries (i.e. a .com, .co.uk, .com.au and .ca) and is serving both duplicate and unique content across each, what is the best practice for the pages on the multiple domains, assuming the company is unable or unwilling to canonicalize the shared content on one domain (for segmentation purposes)?


Great Q randfish, I second this question Danny.

[Wilksy]

I don't really expect answers but I really wonder how Google views Search these days.

Are Google ever going to focus on creating a better way to rank sites (and get back to their core business - why people actually go there), or are you like any good seo and only optimise to a point then jump as soon as a compeditor get's near. It's been a long time since anything radical has altered the serps with good effect. Do you have any tricks up your sleeve?

Or are Google admitting defeat and trying to make money before the ship goes down to something better.

To be honest I suspect the latter..

I also third what PhilC and others are stating - tell us what your doing (maybe you don't know).

All these latest webmaster relations matt cutts blog, G sitemaps, spam reporting etc. all leave a bad taste in my mouth for some reason. It's a token gesture and people are buying into the Google fear. How about boosting people's confindence in Search (yes we are doing your work still) rather than all the scaremongering, business owners hitting the boards are scared to even add new content and think links will not help, it's not a good situation. :o

So much for organising the world's information eh...

[projectphp]

Can you ask about the W3C and moving some of the Search stuff forwards? Sitemaps, although an open protocol, is not officially endorsed. Neither is the nofollow link attribute and even, heaven forbid, robots.txt things like regex matches. Is workign with the W3C somethign that is likely to happen, or is that an antiquated system no longer valid (like the UN some might say :))?

[Wilksy]

I don't know if this has been asked yet, a little busy to read all the posts in this thread atm, so if I am repeating cast it as a 2nd vote ; )

Will Google Co-Op influence Natural results at any stage in the future?

[traian]

I hope it doesn't sound too impolite, but:

1. Do you have you own website that you promote it?
2. Did you ever told "tips" to some friends that asked for?
3. Did you ever try to optimize webpages? What were the results?

Thanks Danny,
Traian

[Amit Sharma]

Hi my self Amit Sharma i have problem when google update last time my site PR have 3 and after that update PR is same, i worked with my seriously but its not gain the PR please help me out how can i get the Good PR and backword link.