The Geometry of IR

[orion]

Mike Grehan informed he recently bougth C.J. (Keith) van Rijsbergen’s book The Geometry of Information Retrieval (2004, Cambridge) of with I have a copy. For those interested, this is IR research material which covers

On set and kinds for IR
Vectors and Hilbert Space
Linear Transformations, operators and matrices
Conditional Logic in IR
The Geometry of IR
Linear Algebra
Quantum Mechanics
Probability

It’s a little book but with a lot of “meat”.

Recognized as one of the top IR scientists of the world., Keith elegantly uses a quantum theory approach to IR, combined with Hilbert Spaces, matrix and linear algebra and quantum mechanics. Keith’s book is a kind of interfacing device for other disciplines. Those in other fields (physicists, computational chemists, etc) can see many connections in an interdisciplinary fashion.

I have the privilege of exchanging emails with prof van Rijsbergen. He mentioned to me they have Terrier, an open source search engine and that they have had funded a new network to bring together IR folk in academia and industry in Europe. These are great news!


On other matters, for those interested in historical data and additional sources, David Hilbert was a brilliant mathematician who discovered space-filling curves (used today in storage algorithms). Early in his career, Mandelbrot realized that space-filling curves (e.g., Hilbert, Peanno) are fractals.

This leads me to mention that I'm providing a reference collection: Fractals in Information Retrieval for those interested in researching these and similar topics.


Regards


Orion

[randfish]

Dr. Garcia,

Thanks for this terrific resource collection. There's plenty of late-night reading in this list

I do have a couple questions remaining on the nature of fractals in regards to IR - perhaps these are addressed in the papers, but it would be great to get a succinct version from you here on SEW:

1. Do you consider the link structure of the web to be fractal in nature?

2. Does that fractal nature carry down all the way to a single web site?

3. Does manipulation of results (i.e. link spam) inhibit the fractal model of links on the web?

4. Is it your belief that these fractal qualities are being exploited by any of the major commercial SEs in order to improve results (at the current time), or is this research simply too new?

[orion]

Thanks, Rand, my good friend.

Before proceeding any further, Keith’s book is not about fractal geometry, yet he uses Hilbert Space, which is a standard fixture in many disciplines. It was Mandelbrot, who later on realized that most of David Hilbert research give rise to fractals. Back in Hilbert’s and Poicare’s times these were called math aberrations and useless anomalies. We know now this perception was far from being accurate.

Another things to remember is that the terms “self-similar, self-similarity, scale-free, scale-invariance, scale-independence, power-law scaling behaviours” and similar expressions all refer to fractals.

I’ll try to address your questions briefly.

1. Do you consider the link structure of the web to be fractal in nature?

Yes and according to
Berners-Lee
http://www.w3.org/DesignIssues/Fractal.html
Kleingberg
http://www.trnmag.com/Stories/090600...ets_090600.htm

Andrei Broder
http://www9.org/w9cdrom/160/160.html
Tomkin
http://alme1.almaden.ibm.com/cs/peop...fs/fractal.pdf

Barabasi
http://www.ncbi.nlm.nih.gov/entrez/q..._uids=10521342

2. Does that fractal nature carry down all the way to a single web site?

It all depends on how the link structure of the site was designed and
how the semantic structure (and themes, topics) across documents was constructed. For small sites, probably not. For large sites, it depends on the above and on the number of length scales involved. Many sites are not self-similar in connectivity or semantics.

3. Does manipulation of results (i.e. link spam) inhibit the fractal model of links on the web?

In reference to links manipulation, is hard to tell. It all depends on which specific link structures we are dealing with. Are we talking about pre-patterned link networks that replicate themselves at different length scales or are we talking about randomly dropping links all over the place? Fractals are not completely random processes, thought they may look random at some lengths scales.

4. Is it your belief that these fractal qualities are being exploited by any of the major commercial SEs in order to improve results (at the current time), or is this research simply too new?

I don’t know about being already exploited commercially by any of the major commercial SEs in order to improve results.

I do know from personal experience and by beta testing software that
fractal technology is being used to organize/manage large repositories of information and for datamining.

Regards


Orion