PDA

View Full Version : Searching Frames

Jan0226
05-31-2006, 03:34 PM
I am new to web development, so take an easy on me, please. Have some questions in regards to search engines. I was doing a site in a frames structure and was told that was really bad for search engine purposes and the search engines will not search the frames. Is this so?

My thoughts are, if that is so true, then why do almost all web development tools, at least the ones I've looked into, have frames available to develop your site with? I've been told frames are more of a concern if you have earlier versions of web browsers, but now a days, it isn't so much of a problem today. I've also done some research and it appears to me that the most commonly used search engines, such as Google, will now search a frames page.

I guess, I'm looking for some advice. Any feed back would be helpful. Thank you!
Brian M
05-31-2006, 05:12 PM
Frames were a brilliant idea back in the days before high-speed Internet access was first introduced, because it was much faster on a dial-up modem to load the outer navigation frame one time and then load interior frames as needed.

However, the search engines have always had a problem with evaluating internal frames, and rarely (if ever) rate an internal frame very highly in the SERPs. Since the outer frame usually holds the page title and other META tags, even if you have one-thousand pages in your site, all pages in your site will have the same title and description in most search engines. The exception is with engines that read and index the internal frames, but then the page titles and META tags are often mixed up, so it makes it very difficult to SEO a site.

Frames are still supported by most browsers, and the development tools are still available, so if you don't care about the natural SERPs and most of your visitors will be using a slow modem to access your site, then frames are still a viable use of the available technology.

Each search engine handles frames a little differently, but there is absolutely no valid reason to use them if your primary goal is to achieve high ratings in the natural SERPs, where most visitors access the web via DSL or Cable modems.

Brian M

P.S. Welcome to the SEW forums!
simons1321
06-01-2006, 09:01 PM
Actually its more because of the html markup that a framed site presents to a spider.
For example your framed site probably looks something like this to a spider:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Your Page Title</title>
<meta name="keywords" content="your,keywords">
<meta name="description" content="Your Description.">
</head>
<body>
<frameset rows="110,*" onLoad="loadDoc()" frameborder="NO" border="0" framespacing="0">
<frame src="your-navigation.html" name="topFrame" scrolling="NO" noresize>

<frameset rows="100%,*" cols="810" framespacing="0" frameborder="no" border="0">
<frame src="your-main-frame.html" name="mainFrame">
<frame src="UntitledFrame-2"></frameset>


</body>
</html>

or something similar to this, this is actually what one of my clients pages looked like before i got to it.

While a user of the site actually sees the frames and images and text of your website, this is what a spider will see in most cases. And since the only thing the spider has to go off of is the Title and Meta tags to figure out what your site is about and no content or links, it therefore gives your site an extremely low ranking if any.

Next time you find a framed site (real estate agents almost always use frames since they want to provide access to the MLS to their visitors), just hit the view cache button on your google toolbar and all you will see is a completely blank screen. Thats because the only thing G or Y sees is the above code which contains no content or images or links.

There is however a workaround for this that works fairly well and still gives you the ability to use frames on your site. The "trick" is to use the <noframes>Insert body content of your page here</noframes> code on your site below the frames tags. In addition to this, slap on some javascript which forces the navigation frame to be loaded on every page and you can easily rank highly in G.

I have two clients which i have employed this tactic and it works beautifully.
PhilC
06-01-2006, 09:28 PM
Sorry, people, but the advice so far has been wrong.

Search engines don't have a problem with frames, and they crawl the frame sources and beyond quite happily.

There is no reason for the inner pages not to have their own Titles and meta tags, just like any other pages. They are just normal HTML pages.

So crawling, indexing and rankings are not a problem. The idea that frames are a problem stems from many years ago when search engines didn't follow frame sources.

Google doesn't index iframe sources, but I don't know about the other engines.

However, there is a problem with frames and search engines that needs to be dealt with. When an inner page is listed by an engine and somebody clicks on it in the serps, the person lands on the inner page. But the page should be in the framed environment, and it isn't. Such pages are usually not a lot of good for people, because they can't wander around the site. When it happens, which is most of the time, there needs to be a system where the whole framed site is pulled together with the required landing page in the main frame. I use javascript for it on www.holidays.org.uk (that site currently has ~16,000 pages in Google's index - all but one of them are inner pages, of course - inside frames). Not everyone has javascript on, so it's not 100%, but I'm content with it.

Here's an example of a URL from that site that might be clicked on in the serps. Click it and see what happens:-

http://www.holidays.org.uk/php/accommsrch.php?type=county&area=Cumbria&cat=A&page=
Brian M
06-03-2006, 01:28 PM
Sorry, people, but the advice so far has been wrong. Search engines don't have a problem with frames, and they crawl the frame sources and beyond quite happily.
Hmmm... If you enter " site:www.holidays.org.uk " in Yahoo, it only has one, single page in its entire index (do not include the quotation marks). The same result appears with or without the www preface, so it certainly appears that this search engine is unable to crawl the frame sources.

Google and MSN also show odd results, with a lot of duplicate pages listed, so this site appears to have been online for a long time in its present configuration.

So, Jan0226, if an expert with as many years of experience as PhilC is unable to successfully get a site crawled correctly by ALL of the search engines, even after resorting to JavaScript, perhaps you might want to avoid frames.

Then again, if more sites used frames, there would be a lot less competition for the top spots in the SERPs...

;)
PhilC
06-03-2006, 02:34 PM
That's not right at all, although I do understand you not being pleased that I contradicted you, Brian, but I'm speaking from experience ;)

Google shows over 15k pages indexed - that backs up what I said.

MSN shows over 7k pages indexed - that also backs up what I said.

Yahoo! only shows one page indexed because Inktomi banned the site long before Yahoo! bought Inktomi. In fact I was surprised when I found even one page indexed a while back.

All of those engines are well able to index frame sources and beyond, as I said.

Incidentally, javascript doesn't have anything to do with getting the pages indexed - they are indexed normally. You completely misunderstood what I said about javascript ;)

Btw, there are no duplicate pages indexed by any engine that I'm aware of.
simons1321
06-05-2006, 01:27 PM
Phil,

In G's cache of your example pages, it shows the content which has been placed in the noframes tags. If my advice on using the noframes tag is incorrect then why is the noframes tag used here and why is it the only thing that is crawled by G???

I was always under the understanding that G can't crawl framed pages correctly, so using the noframes tag is the best if you must use frames. Someone also once told me that yahoo doesnt list framed sites at all cause it cant crawl them properly either.

I'm just a little confused now as to whether or not i should take down the noframes tags on some of my sites to reduce the amount of code.
airtravelcenter
06-05-2006, 02:14 PM
According to GOOG, it is helpful if you include each source page in the sitemap. The noframes code is also effective if you limit the html to that information you would want someone to see when browsing without frames or javascript support. Just enough to explain what the person is missing and link or links for them to get it.
PhilC
06-05-2006, 04:13 PM
There's nothing wrong with using the noframes tag, Simon, and there's everything right with it, but your post indicated that the frameset page is all that a search engine spider will see of the site, and that's not correct. If I misunderstood your post, I apologise.

... and why is it the only thing that is crawled by G??? It isn't the only thing that's crawled by Google - they have over 15k pages of that site in the index.

I'm just a little confused now as to whether or not i should take down the noframes tags on some of my sites to reduce the amount of code.Definitely don't remove it. It's *very* useful for rankings for the frameset page, which is usually the homepage - the page that is most linked to from other sites, and often the page that gets the best rankings.

Theoretically, the content of the noframes tag should be an alternative to the pages that are loaded in the frames, but that's not really viable, because the site's navigation is usually in a frame and, unless special pages are created, the navigation won't be seen again. The best way to use it is to fill it with content about the site, and optimise the content for the site's main topics. My site's frameset page ranks highly for the topic's most generic searchterms, but most traffic comes to the inner pages from the long tail. That's why a system of pulling the site together, with the target page in the main frame, is essential for framed sites.
dannysullivan
06-06-2006, 10:33 AM
As a reminder, I also have my Search Engines and Frames (http://searchenginewatch.com/webmasters/article.php/2167901) tutorial here on SEW that also illustrate why, like with the code snippet above, search engines have a tough time with frames. You'd think they'd have solved that by now, but even if so, you'd still have issues about pointing people outside the frameset.
PhilC
06-06-2006, 12:23 PM
Danny. The tutorial is interesting, but it doesn't tell the whole story. Which engines don't follow src URLs? Google specifically stated that they do. Some minor engines may not, although there has never been a reason for any engine not to. Frames have never presented any problems to search engines, but in the old days they chose not to follow src URLs, and that's where today's thoughts stem from.

There is also something missing from the tutorial - it shows no way of having a person land on an inner page, and have the whole site pulled together with the required page in its frame. The "home" link works to some degree, but most site owners would prefer it to look good, which means pulling the whole site together with the required page on show. The methods in the tutorial work fine if the inner page is the top or nav frame, but not for most pages that belong in the main frame - the most likely ones to be clicked on in the serps. Maybe I should write an article about it.
rainborick
06-06-2006, 09:41 PM
I'm with Phil. This story about search engines not indexing frames-based sites has always seemed to be something of a myth to me. It may have been true before the millennium, but I haven't seen any evidence to support it in the past 3-4 years. Frames are HTML 2.0, for pete's sake. That's circa 1994-96. When you consider how trivially easy it would be for a search engine programmer to simply set the crawler to treat a <frame> tag just like an <a>nchor tag, it makes me very skeptical that any search engine crawler would not have this capability. My pet theory is that early SEOs couldn't explain the technical problems caused if a user clicked on a SERP link to a page that was designed to be contained in a <frameset>. and so they simply told their clients "search engines don't like frames", and thus a legend was born.

In any case, mitigating the problems caused by <frames> are simple enough as well. You set up a <noframes> section with a well-formed, keyword-rich mission statement, and links to the most important pages on the site, starting with the sitemap page. Then you install a piece of JavaScript on all of the pages that automatically reloads a page into its proper <frameset> page and a <noscript> section with an explaination of the situation. Just be sure you get a smart script and avoid the ones that simply redirect the user to a default main page. Sure, some 7-10% of users run with JavaScript disabled, but a client that insists on using frames has to understand the costs.
PhilC
06-06-2006, 10:04 PM
My pet theory is that early SEOs couldn't explain the technical problems caused if a user clicked on a SERP link to a page that was designed to be contained in a <frameset>. and so they simply told their clients "search engines don't like frames", and thus a legend was born.That's something that never occured to me. Another theory might be that search engines knew that framed sites normally didn't pull themselves together with the required page on show, and so they chose not index inner pages, even though there's never been a technical reason for them not to do so. It's always been just as easy to extract a URL from a frame src attribute, as it is to extract one from an anchor href attribute, and once a URL is extracted, it's just another URL to crawl.
dannysullivan
06-07-2006, 07:48 AM
Frames never a problem? I didn't sit down and write a tutorial because I had nothing better to do. They were and still can be a problem. I've seen that first hand from way back when I did this. Also, I agree they might not have been a problem technically for the search engines to overcome but absolutely, they were something many of them chose not to address.

Yes, they could find their way through to frame source pages if they want to. Perhaps Google is doing this now. I haven't tested that for ages. Google gives some suggestion they'll return a frame and restore the frame set here, http://www.google.com/support/webmasters/bin/answer.py?answer=34445&topic=8522. But then again, they clearly don't suggest it's going to work all the time and point right back to my own tutorial for webmasters seeking advice.

I agree with you -- they might still get in an index individual frames, and then you're stuck with the problem of restoring the entire frame set. I do address that in the article suggesting some Javascript solutions. But if you dive deep on an article for just this issue, let me know. I'll get a link added.

Instead of us all going around on whether frames are a problem or not, here's the easy solution. Go find some big sites you know uses frames, then check and see how well indexed they are. If I have a chance, I'll poke around again. But I know from the past in talking with people, many have said that when they dropped the frames, traffic went up. Moreover, if you want to build links, having your best contact be unable to be bookmarked easily (and thus linked) is a problem. I've yet to see a site where I thought "thank goodness they use frames." There's always been another good way to do what they want and make both humans and spiders happy.

So to spin back. No, it's not fair to say frames will keep you out of a search engines, no do I say that. It is fair to say that frames may cause search engines problems, and you'll want to consider the issue before deciding to make use of them. And if you are making use of them -- and if you also are finding your search traffic doesn't seem right -- then perhaps frames are the problem.

Oh, I do have a quick test to show how frames are immediatel a problem for plenty of people:
http://www.google.com/search?q=%22this+site+uses+frames%22

Look at the descriptions. That alone shows a reason to at least consider what you put in the noframes area.

OK, descriptions change based on what you search for. Fine:
http://www.google.com/search?q=mca%20chicago

Look at the MCA Store. Super attractive description, eh?

Definitely, the site is crawled. site:mcachicagostore.org gives you 245 pages (no idea how many in total there are, but it's better than one). As we've covered, the frame context isn't restored (and interestingly, you hardly miss it). So frames in terms of crawling weren't an issue for this site and in fact for most of the sites I've seen -- so kudos at least to Google, they've got the crawling much improved. But it's still something you should keep in mind.
Chris_D
06-07-2006, 08:07 AM
The biggest issue with frames in the 'links are King', and 'Anchor text is queen' era is that if you consider a traditional framed site - the Site navigation is generally in the top framed section; or the side framed section.

So just to keep this simple - lets have a frameset with side nav & framed content - lets call them "parent frame" (with side Navigation); and "child frame" (content without side navigation).

Lets now assume you have say a 500 page website - and the individual 'child frame' content pages are all indexed by the major SEs (and then rebuilt into the frameset using Javascript when a real human using Javascript clicks on the indexed page).

Ok - so how many of those 499 content 'child frame' pages have links to the rest of the site? How many of those 499 'child pages' have external inbound links to the 'child' URL?

Nothings impossible - and you can make a framed site work - but for the vast majority of framed sites - losing the frames and moving to includes etc. using CSS styled UL based Nav and retaining the 'child' URL foir the content pages seems just so much easier and more effective....
PhilC
06-07-2006, 09:50 AM
Bookmarking is definitely a bad aspect of framed sites, as is deep linking from the outside.

Titles and Descriptions are the same as any other pages in the serps, and noframes are fine, but it's up to the site owner to do them properly.

Internal links from "child" pages are also fine, but not all framed site owners think of it.

On the whole, a site is better done without frames, for the reasons stated in the latter part of this thread, but framed sites are fine with search engines, and can work well if they are done well. The owner needs to accept that they won't work well for people who have frames and javascript off.

I haven't created a framed site for many years, and unless there is a compelling reason, I won't create one again. But if there is a compelling reason, I would certainly do it, and it would perform the same as any other site in the engines.
scrubs
06-07-2006, 12:12 PM
The problem that PhilC stated here is what we also experienced with a client. We also used a Javascript to pull through the main frame everytime you load that inner page which may be indexed.

IMO I would suggest working without frames, but in some cases like PhilC above you have to make do with what you've got. In that case there are ways around making frame structure friendly to search engines.

Non-Framed websites are much cleaner and easier to work with but there are lots of dated frame sites still kicking around doing well. I believe using frames will not give G the prettiest picture of your website in the index.