Proposed Search Engine Standard For Titles & Descriptions

[dannysullivan]

It's becoming more noticeable now how search engines may pull your title or description from the Open Directory. And I'm finding myself more and more annoyed by it. I kind of feel like if a search engine wants to generate a title and description from my pages, it should come from -- well -- my pages

Interestingly, we have a long history of this already happening. Yahoo long, long has substituted its own titles and descriptions from the Yahoo Directory for pages showing up in its crawler-based results. We also had history of LookSmart titles and descriptions being used in the former Inktomi results, in place of actual page content.

For whatever reason, Google's greater use of Open Directory data to replace content derived from the page itself is making me personally rethink all this substitution. Certainly we've had a number of people confused by it. What do you think -- OK for this to continue? Time to regain control?

I was out at Yahoo two weeks ago and talked with them about the issue. They were wondering what I thought the solution would be. Personally, I think it goes something like this:

1) Always use the page's actual title
2) If the page has provided a meta description tag, NEVER go beyond the page to create a description
3) Consider the meta description tag strongly as the description to be returned, especially if the search terms for the query appear within it
4) If the meta description tag seems inadequate, then form a description by taking text from anywhere on the page.
5) If NO meta description tag was provided, and you don't feel the page has enough content to form a description, then you can go to a secondary source to create a description.

OK, here's my logic. The knee-jerk is to say always use the title and meta description tag. But I know how many times I've appreciated a dynamically generated snippet of text that highlights the relevant part of the page. I think that works for both the user and the publisher, rather than it being the meta description tag over everything.

The meta description tag still serves a purpose. First, including the key terms you feel the page is relevant for should increase the odds of your own description showing up, if the page ranks well for those terms. Second, it says to the search engines that you DO NOT want them to go beyond your page to get a description. In other words, disagree with what the Yahoo Directory or the Open Directory has to say about your page? This is a way to insist to the search engines that they not replace your own words.

What about pages that lack content, such as all images. Ideally, a meta description tag provides the description. But if people forget this -- and they do -- the search engine is then stuck with nothing to use. That's why in this unique case -- no meta description, no page content -- the search engine is free to go outside for alternative descriptions.

Comments, alternative proposals? I've also added a poll to this thread to let you vote on ideas. I tried to guess at all options. You can select multiple options.

Also, for more background on this issue, see:

• Google Showing Dynamic Titles, from rustybrick's site, with screenshots of this type of thing in action.
• Google Dynamically Changing Page Titles, past forum thread on the topic.
• Meta descriptions displayed in Google results?, related forum thread.
• Does Yahoo Directory use DMOZ listings as the supplementary listings?, related forum thread.
• Where's Google get Title for results?, related forum thread.

[Marketing Guy]

Would be good to have an element of advertising creative control over descriptions that appear in SERPs - could bring a whole new element to the game (and would certainly favour creative copy over auto generated nonsense to a certain extent).

My one concern is:

If the meta description tag seems inadequate, then form a description by taking text from anywhere on the page.

It seems such a simple process, but wouldn't this complicate indexing? SE's all offer questionable versions of "relevancy" and "quality" in terms of pages returned for a particular query. So would asking them to determine relevance of Meta description in order to display it (or not) not add quite a fair bit of work? Would it be possible / worth it?

Also, consider the overall impact the SERP-wide copy change could have. For example, looking at a made up top 10 results you may have:

1. Meta Desc
2. Meta Desc
3. DMOZ
4. DMOZ
5. Random snip of text
6. Meta Desc
7. DMOZ
8. Random snip of text
9. Random snip of text
10. Random snip of text

Then consider if all SEO's know they can have what they write on the SERPs description:

1. Sales copy
2. Sales copy
3. Sales copy
4. Sales copy
5. Sales copy
6. Sales copy
7. Sales copy
8. Sales copy
9. DMOZ (he's not caught up yet! )
10. Sales copy

Now in terms of pure "quality" these results are exactly the same sites as the previous ones. But would a surfer see it that way?

MG

[dannysullivan]

It seems such a simple process, but wouldn't this complicate indexing? SE's all offer questionable versions of "relevancy" and "quality" in terms of pages returned for a particular query. So would asking them to determine relevance of Meta description in order to display it (or not) not add quite a fair bit of work? Would it be possible / worth it?

They already do this. Basically, if you do a search, they already look at the meta description tag, Open Directory descriptions, Yahoo Directory descriptions (with Yahoo) and the page copy itself to decide what to use.

And by relevant, that's left up to the search engines. Typically, they seem to define relevancy as "what words were searched for? look on the page and use the first text that contains those words, whether it's in the meta description tag or not."

Then consider if all SEO's know they can have what they write on the SERPs description:

I think you'd have more variety than that, because you can't anticipate or cram all the words you think your page might show up for into a meta description tag. If that were the case, you'd already have the repeating sales copy situation now. After all, the meta description tag is often used if the terms someone searched for are on it.

[Jill Whalen]

Danny, I absolutely agree with your 5 things, and if the engines were in my control, that's exactly how I would do it.

However, at the end of the day, it's their engine, their results, their searchers, and I think it's kind of arrogant of search marketers to expect them to do things the way we want when it comes to the free results.

Of course since we search marketers know a thing or two, the engines would be smart to listen up. But the thing is, they were doing things basically as you outlined them up until recently. There must be a reason why they're testing other ways of displaying results. Why they would want to make their results less useful is pretty hard to imagine.

Unless of course they'd prefer you to click on the "more useful" ads?

[dannysullivan]

I think it's kind of arrogant of search marketers to expect them to do things the way we want when it comes to the free results.

I disagree. OK, I'll qualify to say that I agree no one should expect to dictate to a search engine what they will do with anything on their pages -- paid or unpaid. It's their ink; they'll do what they want.

However, it's not arrogant at all for publishers to want some respect for their content, so say about how they are presented and some consideration about their concerns. It's not a two way street. We let the search engines index our content because we benefit from that traffic. But if we were ever really, really upset with them -- widespread banning would make tapping our pages for their free editorial a lot harder. If anything's arrogant, I'd say it would be if the search engines simply felt like they could do whatever they want without any consideration for publishers. They pretty much can -- but they don't, which is the difference.

I suspect the change has been done in the hopes of making descriptions more useful to users. And as said, I agree with some of these type of things. But I still think site owners should get some degree of control. It's not unreasonable to me to say don't use a third party source to describe my pages.

Heck, what happens if someone describes Greenpeace as "Web site of kooky environmentalists" in the Open Directory. Far-fetched, but it could happen. And then Google ends up describing Greenpeace that way? And then when it does happen, we get the usually shuffling about of "we don't tamper with results, oh, that's just what the ODP says, we don't say that, oh, we can do x, y and z" and then down the line they'll just make a algorithm change to try and squash that.

How much better for a site owner to have a very clear understanding of where the description comes from. I joke about this during my Intro To SEM session at SES. I go through all the variations that might happen for each search engine, speeding up and up and up and ending that the point is, no one can predict or know anything.

That's a bad situation. That's a bad situation to have with publishers that you have an relationship with. The big spinning wheel of the description may be whatever it decides to be just breeds confusion -- not just for publishers, but for users. In contrast, this is much easier to understand. Picture this FAQ for users:

Where Do Titles & Descriptions Come From
Titles and descriptions come from content on the the actual web pages listed, except in rare cases when pages may lack content. In those cases, we may use a title and/or description taken from third party sources, which are the Open Directory and XXX. Be advised that we do not endorse nor review the titles and descriptions in any case.

And for webmasters, similar FAQ outlining say the five steps above.

[Jill Whalen]

However, it's not arrogant at all for publishers to want some respect for their content, so say about how they are presented and some consideration about their concerns.

It's certainly not an unreasonable request. However, I'd say that in order for us to get that for sure, we'd have to pay something.

Because it's all "free" I just can't help but feel that we have no right to really expect anything from them. The whole it's a privledge not a right thing. Would be nice if we all had a right to be listed in the search engines, and would be even nicer if we got to be listed exactly as we wanted to be listed. But unless we're paying for that listing, we're stuck with what we have.

Not that I'd complain, Danny, if you could somehow get them to follow your 5 criteria...I think it would be great. I just don't think it's gonna happen!

[dannysullivan]

Because it's all "free" I just can't help but feel that we have no right to really expect anything from them.

Sure, you have lots of rights. For one thing, you have the right to expect that you are not being unfairly or inaccurately described. If a search engine goes to a third party for a title and description, that can happen. Your recourse shouldn't then be to hope you can get that other place to change things.

In other words, the New York Times might run a free article about you. That doesn't mean they can be unfair or inaccurate with what they write. If they do that, they can (and have) been subject to lawsuits.

So while you can't expect a top ranking, you can expect that the listing of your own site isn't misrepresenting what you do. Search for Hewlett-Packard on Google, and it's fair to say that describing the home page of HP as "Hewlett-Packard Industrial Ethernet" isn't very accurate -- but that's what current happens on that search, because Google pulls the OPD title. It's probably not inaccurate enough to help HP win any legal case -- but it highlights the problem.

[Jill Whalen]

But it's also reasonable (although not accurate) to assume that ODP information would accurately describe a site.

[dannysullivan]

Reasonable, but not accurate. Agreed

That's probably why I'm finding this a bigger issue than with Yahoo in the past. As said, Yahoo has long replaced titles and descriptions in crawler results with its own Yahoo Directory descriptions. OK, but they aren't turning to a third party in doing that. It's still Yahoo using its own voice on Yahoo's site.

With the ODP, Google doesn't run that. And when people raise issue about ODP stuff with Google in the past, the response is, "Oh, that's not us, that's the ODP." The OPD on their site, but OK, I guess we just suck it up.

I guess the main point is that I entirely agree with you that people cannot be expecting things for the search engines for anything, paid or unpaid. You don't have a right to run your ad. You don't have a right to a top ranking. You shouldn't depend on them for anything. We're on that same page

But neither do I think we're in a sit back and take whatever situation, either. Robots.txt came about because site owners had real concerns on how search engines impacted their sites. Meta description came about because the search engines themselves saw value in working with webmasters. Noel McMichael popularized the idea of a search ecosystem where various parties all have interests. Webmasters are a big part of that ecosystem. And it's not just "nice" for the serach engines to respect us on certain issues. It's a healthy, essential part of keeping the ecosystem good for everyone.

[mcanerin]

But it's also reasonable (although not accurate) to assume that ODP information would accurately describe a site

Although I don't doubt that the editors try to make the titles and descriptions accurate at the time of review, many of them are hopelessly out of date, which is unacceptable for an authoritative source, and provides poor relevancy, IMO.

I'm not blaming the ODP for this (directly, anyway), but I think it's unreasonable for a search engine to rely on ODP data unless it takes steps to insure that the information is accurate.

In short, I think the SE's should stop leeching off the ODP and actually invest something into them in order to make the listings they are relying on more accurate.

I don't want this to turn into a YAODPT (Yet another ODP Thread) but the point remains that as long as a search engine is assuming that ODP data is relevent or authoritative, they are responsible for the results of that assumption, for better or for worse.

Ian

[Jill Whalen]

Originally Posted by Danny

And it's not just "nice" for the serach engines to respect us on certain issues. It's a healthy, essential part of keeping the ecosystem good for everyone.

Sounds like a nice perfect world! Obviously, if anyone could help make this happen, you could Danny, since you are our middleman. I hope you can, because I agree that it should be done the way you've laid out. I just don't really believe the engines care enough to do it, as they obviously have their own agenda.

Originally Posted by Ian

I'm not blaming the ODP for this (directly, anyway), but I think it's unreasonable for a search engine to rely on ODP data unless it takes steps to insure that the information is accurate

Yeah, but Ian, it's their engine and if they think that their users will like the information presented this way, then that's what they can and obviously will do.

In the end though, the market will bear out whether they are doing things right or wrong. If people start disliking those crappy ODP descriptions that show up in Google, maybe they'll go somewhere else. (Unfortunately, inertia is a powerful thing!)

I think the engines will keep experimenting and measuring things, and eventually come up with the plan that works best for them; i.e., the one that makes them the most money.

[I, Brian]

Is the ODP simply reliable and consistent enough to be able to provide useful and accurate information on sites for SERPs?

[Jill Whalen]

Brian, certainly ODP is not, imo. But Google's not dumb. They can read just like we can, and they know that ODP data pretty much sucks! They must have some reason why they are testing this.

[I, Brian]

What would especially concern myself would be the elevation of importance of the DMOZ in this respect, almost implying an objective judge of a website.

This is especially of concern as some parts of DMOZ - even non-commercial - are almost impossible to get a listing in, due to lack of editorial involvement, and inability to care to take on new editors.

Would this mean that sites not listed in DMOZ are therefore deemed to be of less important?

A DMOZ link used to be regarded as useful - I'd hate to see it as essential. Not with the current inconsistencies in the directory's development.

[Jill Whalen]

A DMOZ link used to be regarded as useful - I'd hate to see it as essential.

Actually, I think it would have the oppposite effect. I'd be inclined to want to dump my DMOZ listing and not recommend one to clients just as I've been doing with Yahoo listings, for the very same reason!

[mcanerin]

That's a good point, Jill!

I could actually see that as a new SEO tactic - dump DMOZ in order to have more control over the description and title of your website

I wonder if having a strong call to action and a relevant, well written title outwieghs the link pop resulting from an ODP listing for your ROI?

I suspect it might.

I have several clients with top 1-5 positions in highly competitive areas (ie pharmacy, etc) that are not in ODP, so it's certainly not essential.

This seems to be a trend with G, first they dropped the support for the keywords meta, then started ignoring the content in favor of incoming link text, now they are ignoring the descriptions and titles in favor of 3rd party reviews where possible.

I wonder when they will begin ignoring the sites altogether?

Is it my imagination or are they moving towards a meta-directory system where instead of using their own directory, they use other peoples, and forget about actually looking at the sites in question other than for deep links?

Ian

[David Wallace]

I agree with you Danny.

The problem with depending on a directory first of all is that they most typically use the company name or web site name as a title and what is compelling about that?

I think of SERPs as more like book titles with descriptions. Depending on ODP for titles would list a bunch of company names as titles for the listings which tell you nothing about the page that is actually listed there such as a title tag can.

Then add to that how difficult it has been to get listed in ODP or get changes done and it can become quite a mess. I am sure there are great ODP editors but I am also sure that the bulk of them are not quite so active and there is way too much content there for the few editors that are active to keep fresh and up to date.

Therefore I am totally against Google or any other search engine depending on ODP or any other directory for information unless as mentioned before, the web site owner neglects to provide the search engine adequate data describing the page.

[Jill Whalen]

Therefore I am totally against Google or any other search engine depending on ODP or any other directory for information unless as mentioned before, the web site owner neglects to provide the search engine adequate data describing the page.

Well so am I. But guess what? It's not up to you or me.

[gdgrimm]

OK. I've added my vote.

As for my two cents:

The one thing that I'm strongly against is using content for SERP that isn't anywhere on the page it links to.

I'd prefer an algorithm that went like this:
1) Use Title if one is specified
2) Use META description if one is specified
3) If one of those isn't specified, make one up based on other content from the page.

Obviously, I tend to side pretty heavily with page owner control.

[Chris Boggs]

I agree that we as SEM's really have no say as to how the Titles and Descriptions will be indexed. I don't see what the matter is with describing Greenpeace as "kooky." They would probably like that anyway

Seriously though, if a site takes the time to provide a good descriptive Title, it should certainly be used. I also agree with your other five criteria, Danny. The only sticky part comes in the case of pages that do not provide any textual content. In my opinion, they don't really deserve to be ranked highly for a particular search term if the page content doesn't reflect the search. If it's all images then I would hope they are attributed...and they can appear in the image directory then. The whole point of providing results is to provide relevant content to searchers. If the SE has to look further than the page itself, then that page probably does not fit the bill. There are plenty of sites with indexable content that deserve to be ranked.

In my opinion, I also prefer the actual snipets from on-page content, but I guess I can cave and let image pages get away with a Description tag use.