Search: The next five years.

[AussieWebmaster]

Got to love TLAs... well some of us do.

[khawe]

When looking at the future of search you need to look at how SE are treating video. Both Yahoo and MSFT factor in view #'s and content rating into their rankings for videos - along with KW / content relevance. Granted it is not comparing apples to apples but still bring up many interest SEO implications.

[beu]

Signals from end users who previously couldn't vote for content via links from Web pages are now able to vote for content with their clicks, bookmarks, tags, and ratings.

Thanks to Google's Googlebot for RSS feeds, Feedfetcher, "Voting" is possible not only via click, bookmarks, tags and ratings but then again by subscribing to feeds from bookmark tag and rating sites.

SEO will give way to a new form of digital asset management and optimization. This new SEO will place a much larger emphasis on optimizing a range of file types, from PDFs to images to audio/visual.

This was stressed during last weeks factory tour. In 5 years everyone here will know what "EXIF" means. GEO EXIF in particular, has a number of implications for Sketchup and as a result Sketchups integration with Google Maps, Earth and Android.

Personalization and digital asset optimization will end 1999-style ranking reports, as search engine results will be based on blended results from end-user specifics, such as geographic location, time of day, previous searching history, and peer group preference.

At Google I/O last week Marissa Mayer pointed out that perhaps the best signal is a users previous query.


Great article by the way Mike! I think "page load speed" will be the most critical issue for SEM, SEO and PPC during the next 5 years. Fast loading pages are critical for mobile not to mention, slow pages waste search engine resources. To support this idea, I would point out Larry Page's obsession with efficiency, Google's focus on reducing power usage (IE DCs using hydro, solar and wind power alternatives), landing page load time being a QS factor and the fact that Google picked off Steve Souders from Yahoo and seems to be paying him at least in some ways to write version 2 of his book.

Oh well, my two cents! Great thread guys... One thing is for certain, it will be interesting to look back at this thread in 5 years just to see how wrong we are!!!!

[AdrianB]

From a user experience, I would imagine that 'search applications' will become more prominent. There are already tools I use today like Search Automator Pro to help me wade through the shear volume of different media types.

Further, applications like Copernic Agent will become more refined and will allow better management of search profiles and user preferences, e.g. only return pages without this particular type of contextual advertising included ;-)

A 'search aggregator' appeals to me personally as I like to think I'm in control of my searches!

Adrian

[adman1212]

(Adrian B - Welcome, I'm Adrian C!

I figured the Paranoia over Analytics data showed that people do (or did) expect this to be used to some degree. There's plenty there for Google to pick through and do something with - bounce rates for example, are a pretty clear indication of a sites worth for a given query.

I think it somewhat depends on the search engines success at providing quality and democracy - if I all I ever find are pages from big companies or a limited number of sites then I'd probably start looking elsewhere (as I do for some searches). Maybe at vertical search sites, social or just known favoured sites - afterall Google is useless for some searches.

It's all time saving that underlies this - I know if I want football fixtures I'm best off with the BBC, as Google can entirely fail for this type of query. I buy from Amazon as a default choice, not due to price or service - just for ease, because I don't have to enter all my details into yet another form.

I didn't have ADHD before the internet came about.. but perhaps we're all going that way if we spend too long online! I think SE's have been catering to the diminishing attention span for some time.

[Mike Grehan]

beu said: Great article by the way Mike!

Thanks, I have a feeling that this is a good discussion that people can dip in and out of as new thoughts arise. Some excellent comparisons of your own there, by the way.

This is something which really caught my attention:

AdrianB said: A 'search aggregator' appeals to me personally as I like to think I'm in control of my searches!

I took a look over at the Copernic site. I’ve known about this tool forever...And then forgot about it.

But I was stunned to see that it now has over 30 million subscribers. And absolutely, Search Automator Pro seems to be getting a lot more traction with serious searchers by the look of it.

So your post raises this question: What happens if, in the future, we all have a search aggregator to manage our search results by file types and geography and business and personal and...

There is no doubt in my mind that search is predominantly the way that people will find things online. And a useful tool/resource such as a search aggregator is bound to make life easier when it comes to finding your stuff.

But that would mean you wouldn’t need to go to the Google interface or Yahoo! or MSN, or anywhere else for that matter. Your own search bot would be programmed to go out and find results from preferred (or recommended sources) and you also get to choose whether you want white papers, movies, images, blog posts, news etc. etc.

So here’s the big question: Who’s adverts would you see in that mix?

Then, I took a look at Google’s personalized search offering and also Yahoo! The types of services/tools which are being developed may at some point in the future easily be rolled into a Copernic/Search Automator type of widget/gadget/tool. It’s already very close.

Does that mean that, in the future there could be a situation where your chosen search service pulls in results from all over the web, Google pulling from Yahoo!, MSN and those doing the same. So now, you have results from all of the major search engines, but you only see the ads from your chosen search service.

Can anyone else see where I’m going with this? Have thoughts to add on the idea of a desktop search widget that rolls the whole thing together in one set of results that you can edit, save, delete?

And then how do we optimize around that? Just me thinking out loud, as usual.

[AdrianB]

Originally Posted by Mike Grehan

There is no doubt in my mind that search is predominantly the way that people will find things online. And a useful tool/resource such as a search aggregator is bound to make life easier when it comes to finding your stuff...

I suspect the same thing will happen with this that has gone before with social networks, video distribution sites, analytic platforms, payment processors etc., The companies with the deepest pockets will either buy out the innovators or develop a killer app for free.

The difficultly is resisting the temptation, as a user, to wean myself off of Google search results and give some of the others a fair crack. If 'future search' is owned by one company, then that will be a very bad thing indeed for both searchers (manipulated results) and advertisers (ever increasing CPCs).

[bragadocchio]

Some possibilities for search advances over the next five years:

1. Regrouping of universal results by concept instead of content type, for rapid disambiguation of concepts related to a query and better targeted search advertisements.

2. Creator labeled content using something like openID, with metadata citing information such as editor, site owner, content creator, approved syndication locations, etc.

3. Object level ranking for named entities, with more answers shown directly within search results, and more final destination links (like Google's site links) listed under a search result, including results from other sites.

4. Better use of facial recognition and image recognition techniques so that statistical image models can be used to index pictures that don't have accompanying meta data.

5. Better use of user information in search sessions culled from query logs and tracking data from browsing history to aid in query expansion and refinement suggestions for informational and transactional queries, and in determining best final destination pages for navigational searches.

6. Increased use of geolocation information with mobile phones to identify the locational sources of queries, and to automatically tag mobile phone camera images and identify landmarks and locations, provide private shared maps, supply human friendly driving and walking and transit directions with ads for businesses and services along the way.

7. Search kiosks in shopping and resort areas, tied to billboard advertising, displaying business sales and available inventories and coupons and menus and wait times at restaurants, enabling the taking of reservations for those restaurants, and more.

8. Present day and future looking decision-making search tools for business and government and policy makers, based upon information taken from sources like news articles/announcements/press releases, that can enable those decision makers to base their actions on a more informed understanding of the activities that are, and will be taking place around them.

9. A deeper integration of desktop and Web based search, providing automated query results related to the content being used or recently used or viewed by people working upon desktop documents, reading emails, updating calendars, etc.

10. Targeted online advertising based upon actions taken offline, such as credit card purchase, television viewing, attending events, and others.

I could go on, but that's probably enough for now.

[AdrianB]

Wow bragadocchio, that's a very thorough list. I'd hate to be the Sales Executive pitching this to an SEO prospect in 2013!

[bragadocchio]

Hi Adrian,

I don't know if it's a thorough listing. I think my list is ust a small sampling of some possibilities, without taking too great a step away from what may be possible right now.

A number of them could have some interesting impacts on how SEO and marketing on the web is done, and I would look forward to talking with people about them.

[Mike Grehan]

bragadocchio said: I don't know if it's a thorough listing. I think my list is ust a small sampling of some possibilities, without taking too great a step away from what may be possible right now.

Hey Bill... You took your time

Excellent contribution and some great insight. The regrouping of universal results by concept as opposed to content type is something I touched on (here or in my ClickZ column).

My guess is that, as personalized search begins to get more traction, search engines will be able to get a much better picture of the end user search behavior. And then they will be able to tailor results according to the preferred vertical/content type.

Did you have any thoughts on the search aggregator type tools I mentioned. That’s an interesting area to explore. Not just a tool for maintaining and managing all your search stuff, but also your bookmarking, tagging and rating etc.

Thanks for pitching in. Looking forward to more!

(Why don’t you give that Li Evans at the desk next to you a nudge

[jimbeetle]

and you also get to choose whether you want white papers, movies, images, blog posts, news etc. etc.

This past Sunday evening I watched "When We Left the Earth: The NASA Missions" on the Discovery network. Being a child of the 60s I was fascinated by the concept of man traveling into space and watched every launch and recovery (Yes, if something was happening in space, little Jimmy wasn't going to be in school that day.).

I'm still fascinated. On Monday I decided to do a little brushing up on the concepts tested during the three programs that led up to the moon landing. So, simple searches for [project mercury], [project gemini ] and [project apollo].

I was very, very disappointed. With all of the material generated about these three programs over the past 50 years -- much of it media rich -- I was mostly presented a list of links as results.

Google, for all its vaunted universal search, let's get everything in there that the user might be interested in approach, was the worst offender. In only one case (project gemini) did it even include an image onebox. It did include the related searches blurb at the bottom of the results, but by that time I was out of there. Yahoo and Ask did a bit better with related searches. Still no video results for either; Ask at least did have on-page image results. MSN Live? Not even in the game.

What do I want in search results? Well, I guess I want everything. I want all media formats returned. I want them easy to review. I want to select them, sort them. I want to save the sorts as an outline for a presentation or a paper (or maybe just to again relive the excitement of the 60s ten years from now).

One of the things that got me into this Internet stuff was Apple's introduction of HyperCard back in the 80s. It was the first hyperlinking app available to the general public. So, when I saw the first surface computing vids a couple of years ago to me it was the perfect melding: search results returned in discrete stacks, maybe by format, maybe by concept. Clickable, sortable, savable -- usable.

That's what I want. I think. Until something better comes along.

[AdrianB]

What do I want in search results? Well, I guess I want everything. I want all media formats returned. I want them easy to review. I want to select them, sort them. I want to save the sorts as an outline for a presentation or a paper (or maybe just to again relive the excitement of the 60s ten years from now).

Then I think this where the search aggregator applications come in. As mentioned in my earlier post, Search Automator Pro gives me access to all of the major search engines and sort the results quickly in a way that suits me. It certainly isn't a perfect solution but definitely makes my search experience a lot easier.

[bragadocchio]

Hi Mike,

Thanks.

The major search engines have an incredible amount of data about user behavior. The bottleneck may not be so much in collecting that information as it may be in figuring out the best way to make use of it.

Did you have any thoughts on the search aggregator type tools I mentioned. That’s an interesting area to explore. Not just a tool for maintaining and managing all your search stuff, but also your bookmarking, tagging and rating etc.

I've been watching Google slowly release patent filings on something that might combine desktop/intranet/internet searches with alerts/annotations/tagging as well as software download management and many other tools and features only now hinted at in the titles to unpublished patent applications from them. One of the patent documents published at WIPO listed 51 related patent applications in this collection.

You're seeing some of the potential outlines to such an approach in the personalized services from Google, and from Yahoo.

Keep in mind also, that when someone searches at Yahoo, and they have a Google toolbar installed, that Google can see the results of their Yahoo searches, and the selections those searchers make. I believe that Yahoo explicitly mentioned in one of their patent filings that they might do that with their toolbar, looking at search results from other search engines and user behavior associated with those searches.

How do you optimize around that? There's the rub.

Liana's desktop next to mine is actually a virtual one. She's probably closer to you than to me.

[Webmaster T]

Microsoft Launches Beta Release of SearchTogether PluginSeems interesting and on topic. The social aspects of this are more interesting and usable then the academic examples provided in the release.

Ubiquitous/WiMax connectivity will be responsible for many changes to search going forward. Mobile will not be just phones but computing devices that have phone capabilities. This is a much different environment that search will approach in a different manner because the user expectations and reasons for searching are easier to determin.

Research by the US Gov't resulted in the ability to index voice in video. IMO, when that technology is improved (my discussion with them indicated that the voice recognition part of the technology had a long way to go) we'll still have 10 links in the results but many will point at video. IMO, video is coming of age with the increased penetration and affordability of broadband and the high regard videos have with many users.

The capability to index PDF has been around for a long time, but, you seldom see them in the results. Whether that is a result of the amount of content in that format, the degree to which it is SEOed, or a decision by the SE's to dampen the visibility has not to my knowledge been researched. Personally, I prefer HTML to PDF but... that's just me... or is it?

Personalized search is gaining traction and the social network sites do provide some indications of user preferences. Universal search is ever expanding as I see more and more Brand one boxes for sites that didn't have them 6 months ago. Going forward I see the SE's wanting these abilities to improve and drive more of the results.

[AussieWebmaster]

Mike's new article today explores a new angle - Google who many now sees as the internet - or at least the way to get access to all of the internet/web - cannot meet the ever increasing additions to the total....

the crawler as it works today definitely has reached its limits.... as Mike told me last week over dinner - nearly 3 times as much new crawlable content is added to the web every day than is crawled. This growing gulf will be the biggest hurdle the web faces over the next couple of years...

What the new options are I think will be the topic of some future articles by Mr Grehan.

Your comment about the seeming non-existence of companies or people who do not have a web site or web presence is another area that will up for major discussion in the near future and beyond...

[Misscj]

Very interesting responses all round.

I work in A.I research, namely in natural language generation and natural language understanding, after a long stint in IR. We've been looking at machines interacting with data, "understanding" it (it has to be in inverted commas until someone defines what understanding is)and doing something with it. Some people, as well as myself are looking at question-answering systems, which would follow back to those days where you tried out those betas that kinda disapeared and not many people carried on using. Machines don't understand very much yet, and NLG is quite basic still, despite great efforts. It is however advancing. The pattern recognition and keyword retrieval based methods work to an extent but aren't scalable. This in my opinion goes for search enginges, and the work in digital libraries too. We're working on A.I techniques. So is everyone else

I guess what I'm saying is that I see machines playing a much greater part in our experience of the internet. Machines creating documents, creating summaries of groups of documents for you, can we we do away with ranking altogether please? Can we have data with answers that we're looking for rather a list of documents, it's hit and miss whether it's the right one and, I don't know about you but I have to go through a few until I have gathered all of the information I need. Can we bypass this? In time, yes I think so.

Accessing documents for research and information access without visiting a site, just through explaining what you want, in your own words, and getting a thorough response back, allowing for further refinemend through your own language. Sending out a machine to buy stuff for you? Maybe.

As we now shop from home, could we shop without window shopping for answers on the net? Maybe, maybe...maybe not soon, but maybe.

Machine translation and speech are big projects right now. Both need A.I techniques too, and when it comes down to it, all of the techniques in NLG, NLU, NLP, IR come together.

Another issue with developments here is gathering user data to allow for testing and learning. Privacy is an issue. I need to gather online conversations, and it is hard work getting people to let me access those.

btw, if any of you would like to donate your chats to me...don't hesitate to contact me

[Webmaster T]

Mike, great article and astute observations on crawlers ability and scope. Do you believe the dropping of supplemental results in the current Google SERPs are a step toward decreased crawling of that index? Do you believe the recent Google announcement of executing forms to uncover content was to find previously hidden content or just an excuse to look up a sites skirt?

My thinking has been that the supplemental index is being dropped to free up crawling resources for the new content they'll find behind the forms. I also believe that all Google crawling, including the media crawler, are contributing to the organic index.

IMO, if you don't refresh that supplemental index very often you have freed up a lot of crawling resources. I know I see large decreases in what's being crawled on large dynamic sites.

[Mike Grehan]

Misscj said: I guess what I'm saying is that I see machines playing a much greater part in our experience of the internet. Machines creating documents, creating summaries of groups of documents for you, can we we do away with ranking altogether please? Can we have data with answers that we're looking for rather a list of documents, it's hit and miss whether it's the right one and, I don't know about you but I have to go through a few until I have gathered all of the information I need. Can we bypass this? In time, yes I think so.

Misscj - Nice to have someone else new in the thread.

I must have written about this so many times. You know, when you had a dial-up connection and limited bandwidth, ten blue links (which as you say, some of which may be relevant and some won't) was a decent enough experience. But in this day and age it's just a suboptimal return.

For those who were present at Andrew Tomkins (VP, Search Research, Yahoo!) keynote at SES, New York, you'll have heard him voicing similar concerns. At one point, he stated something along the lines of how "scarily similar" the SERPs looked - ten blue links - ten years ago.

And apart from anything else, here we are still talking about the only way a document can rank is if people have links to point to it. No wonder spammers don't want the ten blue links to go away. They'll be out of business.

Hey search engines – did you hear that?

Anyway, the crawler thing... The bare basics, that I wrote about in my recent column are based around this. If (and these figures are out of the blue no research, purely as an example) a search engine such as Google downloads 30 million new pages a day and content creators (and end user generated content) hits 90 million pages a day, the math is not too hard to figure.

Let's take this new search medium and measure it against an established medium such as TV. A catastrophe happens somewhere. User generated content appears on news channels world-wide within hours.

In search? A day? Days? Weeks? It all depends on where it appears on the web first.

Maybe tag-clouds and bookmarking and rating are also slow to the mark (although I do believe they carry a little weight for certain types of query).

So, anyone else have thoughts about the crawler and the tightening limitations on what it can bring to the surface in the SERPs?

Webmaster T said: My thinking has been that the supplemental index is being dropped to free up crawling resources for the new content they'll find behind the forms. I also believe that all Google crawling, including the media crawler, are contributing to the organic index.

Personally, I think the supplemental index hasn't changed. It's just not being flagged anymore. And as we know, Google is under represented (the crawling thing) for certain queries, so the supplemental creates a neat little back fill. Rubbish, but a back fill.

[AdrianB]

I mentioned in a previous post about search aggregators but a colleague has just flagged up rollyo.com to me. Apart from being deeply embarrassed about something else that seems to have passed me by, could this fall into the 'search in the next 5 years' category?

My first impression of this was 'wow' followed by a 'oh' as is 'oh, what will the major search engines do when their ad revenue goes down because the searchers are pulling in organic results without 'their' paid ads?'.

I'd be interested to hear of other's experiences of Rollyo and opinions for where it fits in the future of personal search.