Future of Search Engines: Cooperation with Web Site Owners

[AlexTajirian]

Success for a search engine has two mutually enforcing components: how many searches are done using the engine, and how useful the results are to the searchers. Search engines have focused on algorithms to improve search-result ranking and to determine the best ad for a given text content. What they have not done is take into account the human element, namely how different searchers and Web site owners may feel about content and “look and feel”—the searchers’ various preferences and the owners’ different beliefs about what does and doesn’t work. In “Search Engines Must Incorporate Private Information in Search Result Rankings,” I discussed why it is necessary to incorporate such private beliefs regarding the relative quality of a page’s content for a key word search. In this essay I postulate that incorporating information about searchers’ preferences benefits not only site owners but all the parties involved.

Search engines cannot improve relevance of results unless Web sites do their part. The search engines can identify searchers’ preferences through questionnaires, but then Web sites must incorporate the findings. Of course, Web site owners benefit from customized pages whether or not search engine algorithms take the pages into account. Everybody wins—Web sites, searchers, and search engines—when result links lead the searcher to a customized page on the destination site. The cooperative benefits are in addition to any customization features provided by search engines to users.

Preference customization has come a long way, but it’s still in its infancy. One of the interesting approaches is cognition-based algorithms that go beyond those based on demographics and (see Glen L. Urban, et al., “Morph the Web To Build Empathy Trust and Sales,” Sloan Management Review). Click-path analyses drive the cognitive-based algorithms, which have the additional advantage of inferring visitor preferences without any effort from the searcher. It is time for corporate Web sites to start incorporating such analytics and sharing them with search engines.

The key to remember: Future cooperation between searchers, Web sites, and search engines benefits all parties.

[sitetruth]

That tends to turn into "pay for placement", which got search engines in trouble with the U.S. Federal Trade Commission the last time it was tried.

[AlexTajirian]

Good point sitetruth.

Paid inclusion has failed for a simple reason. If the advertiser is paying, the service provider has to accept the advertiser’s wishes at face value, irrespective of quality relevance to the associated key word. Advertisers do match Web page content with relevant key words, but over time the informational quality and relevance of pages have diminished because so much of the content has been pure advertising links on domain parking pages.

In the future, when a free cooperative mechanism is in place, the information provided by Web sites will be one variable in search result algorithms; thus, quality will not be compromised. Moreover, Web sites will have an incentive to reveal their true beliefs about key word–related quality, as they would otherwise face revenue loss and the threat of being excluded from indexing.

Thanks for getting me to think in new directions.