View Full Version : Grid computing
So far I have not seen any evidence of the SEO community feeling threatened or concerned about Grid computing which is positively a big deal in research circles.
It is fast growing to move from a purely research based project to a more widespread endeavour. this reminds me of the beginnings of the internet. Companies like Microsoft, sun Microsystems, IBM, have all been investing in the technology. Sony is incorporating grid computing into the Xbox 3 to enable a better multiplayer gaming platform. Butterfly.net is looking to use it for gaming,... Microsoft is investing heavily, ...
We're talking about being centred on information provision in its pure sense. Operating systems could also be distributed. Globus is currently available for a test run through its toolkit.
Speculation only:
- the grid can be used to share information
- As it develops it is likely that the grid matures to such an extent that it is able to provide more centred information to the user by directing him/her to the correct area, or group of relevant servers.
- this information may be much more interesting than websites which offer many things distracting the user from the original aim.
- So far about 16% of the web is indexed and we have no idea how big it is, the grids solve this problem.
- the idea of the semantic web fits nicely with the grid concept as information can be picked up in an informative way to be class the server in a relevant category.
- SEO has to find new ways of adapting to the grid.
Please tell me why this issue has not yet reached the SEO community as it has ours. It is still very very young but has many possibilities.
rustybrick
02-22-2005, 09:58 AM
I am sorry, but maybe I am not understanding.
Grid computing is to enhance "cooperation--ablity to share and agreegrate distributed computational capabilities and deliver them as service." source (http://www.gridcomputing.com/gridfaq.html)
Doesn't this mean that it just enables the engines to work at greater speeds?
Maybe do queries a bit faster?
I mean, do you think that they can process more complex algorithms to return more precise results because of this allocation of server resources?
I just mean that the implications of such a system are very interesting. Some are calling it "internet II". If we think about how the p2p network functions, we can start to understand how boosting processing power through a grid could change the way we search, as instead of looking for sies, we might look for hubs of information characterized by server homes.
Its all speculation as the grid is a baby, but there's possibilities there don't you think?
rustybrick
02-22-2005, 10:04 AM
Hmmm....
There are tons of possibilities in search in general. As you know, there is so much more that can be done.
So, does grid computing help it along? I assume. Does it offer something by itself? I do not know.
Isolating grid computing, what do you see xan?
I see lots of speculation, lots of large investments by major players, and a lot of research and information at this point, but nothing telling us why everybody is investing so much.
WilliamC
02-22-2005, 10:21 AM
<NevDull> All grid computing is is the ability to use compute cycles on demand regardless of where they are.
Xan, can you be more specific about what the heck you are talking about possibly happening? As most see it at this point is basically paranoid delusional fantasy. Maybe if you clarify...
I think perhaps "paranoid delusional fantasy" is a bit harsh, mind you they said the same about the pc.
"In the early stages of the evolution of wireless telephones, nobody expected that the technology would extend to Web browsers and cameras, or that mobile telephones would morph into a combination of telephone and personal digital assistant (PDA) functions. It is this kind of innovation and subsequent evolution in the application of grid computing that we will experience.
Today's challenges and main applications of grid technology barely hint at its full potential."
(MCNC)
Since the Internet is a network of computers, often waiting for work in their spare time, why not treat the Internet (or some subset of it) as an extension of the enterprise, farming out number churning, customer management, expert services, even certain special kinds of thinking to other companies? And possibly this kind of exchange will lead to new kinds of companies -- or at least to a new way of looking at your own company. (IBM)
Ian Foster, senior scientist and head of the distributed systems lab at Argonne National Laboratory, Argonne, Ill., is widely recognized as one of the top sources of information on the grid computing movement.
Ian Foster: The open grid services architectural (OGSA) effort is working to take grid technologies and integrate them into a Web services framework. In a sense, that work is integrating management capabilities into Web services.
A snippet:
Currently, a "second Internet" is in development at higher education institutions across the nation. Coined the Internet2 project, the goal is to develop a bigger, better, faster, more secure network on which to develop and test advanced networking technology.
a link for internet 2 (http://www.internet2.edu/)
Thank you Sun (http://www.sun.com/products/hpc/pdfs/grid.pdf)
"The "Grid" has the potential to become a global web of ubiquitous electronic services which will improve infrastructure utilization, increase data access / integration, enable new levels of communication, and provide for the creation of new applications.
The grid is the product of evolution, dovetailing the next generation of the Internet and the World Wide Web, an advanced form of today's Web. The grid will be used for computing, collaboration, and communication. And like the current Web, the grid will be ubiquitous. It will simply become the basic IT infrastructure for many businesses and applications, within research groups and commercial enterprises, and, eventually, for consumers."
(Grid computing & Network services)
Of course its speculation, but so was the internet, and so was Google, and so were many many scientific advancements. Speculation is at the heart of creativity.
What I wanted to discuss was the impact of search services on something that is built different, something that does not follow the same rules and could quite possible mutate to a new kind of internet, where results and so forth are not delivered in the same way.
NevDull
02-22-2005, 12:13 PM
So far I have not seen any evidence of the SEO community feeling threatened or concerned about Grid computing which is positively a big deal in research circles.
Sure it's a big deal in research circles... it allows lots of computing power to be used as needed on-demand. This is something that researchers need.
It is fast growing to move from a purely research based project to a more widespread endeavour. this reminds me of the beginnings of the internet. Companies like Microsoft, sun Microsystems, IBM, have all been investing in the technology. Sony is incorporating grid computing into the Xbox 3 to enable a better multiplayer gaming platform. Butterfly.net is looking to use it for gaming,... Microsoft is investing heavily, ...
Yes, Sun and IBM are investing heavily. Why? To make money. They sell hardware and software to run clusters on. IBM is selling a toolkit which extends the Globus toolkit, a bit more open than Sun's "Grid" software which isn't designed to be interoperable. Both sell computer systems which can be clustered on a large scale. Sun isn't selling so many $1M+ systems and certainly fewer in the $200K-$500k range, so they're selling lots of $20K systems and tools to utilize them towards a common task.
Microsoft feels left out in technical computing, so they introduced a distributed computing toolkit. It's an API, not a "Manage a grid" application.
If you're talking about the PS3's Cell processor, then calling it in itself a grid is silly, though there are single chip multiprocessing capabilities which are interesting approaches to continue to see processing speed increases where the MHz race has stopped, or at least stalled.
We're talking about being centred on information provision in its pure sense. Operating systems could also be distributed. Globus is currently available for a test run through its toolkit.
No, we're not talking about information provision. How does one provision information? We're talking about compute resource provisioning on-demand and location-independent.
Speculation only:
- the grid can be used to share information
As can any network... and there isn't, and never will be, only one grid. People won't give away what they can charge for... there's nothing preventing multiple grids... and there are competitive advantages for a number of different players who will have their own grids.
It's never going to be a "sit around the campfire holding hands" kind of thing.
- As it develops it is likely that the grid matures to such an extent that it is able to provide more centred information to the user by directing him/her to the correct area, or group of relevant servers.
You're assuming a single distributed grid. This is wrong for many reasons, including commercial interests and international politics.
- this information may be much more interesting than websites which offer many things distracting the user from the original aim.
Computers don't make up information, they process it. There is absolutely no basis for this statement.
- So far about 16% of the web is indexed and we have no idea how big it is, the grids solve this problem.
How does this solve the problem?
- the idea of the semantic web fits nicely with the grid concept as information can be picked up in an informative way to be class the server in a relevant category.
Content doesn't reside permanently on any server in the grid. So how do you propose this works?
- SEO has to find new ways of adapting to the grid.
I think you have a fundamental misunderstanding of what grid computing means to the Internet as a whole, and I think you need to find new ways of adapting to the grid. ;)
Please tell me why this issue has not yet reached the SEO community as it has ours. It is still very very young but has many possibilities.
We deal with optimizing the search, for the hundreds of millions of end users, of information for which there is an established hierarchy and searching mechanism. Grid computing does not fundamentally change anything about that information hierarchy, so there's nothing for us to focus on optimizing. When there is something to optimize for, it will be done.
-NevDull
Hi NevDull,
Did you read the snippets I posted above?
I do understand the grid very well, I work with it. I do know exactly what you're saying, but its not really the question. The question is can you look further than that?
The sources I quote are centered around the future and the internet, etc ...
I mention p2p as well. That's information provision right? Why then believe that the grid will just be about hardware and processing power?
"Computers don't make up information, they process it. "
No you think? I thought computers were the ones building websites. You mean its people who do? Damn...there goes years of research.
All i'm saying is it isnt smart to be short sighted. It doesn't mean that any speculation is ocorrect either. Its just a discussion.
Yes the internet needs to move on, and there are sevral ways it can, this is one.
You are not being forward thinking enough about this to see what I'm getting at. I do know what you're saying but I didn't set out to give a lecture on grid computing, I assume there's plenty of information out these for people to refer to.
I think you are talking about the near future which is not where I'm going with this.
NevDull
02-22-2005, 01:53 PM
You're talking about the future in the same way that the people selling vitamins on infomercials do. All hype, no substance. Yes, maybe the Internet will change dramatically. But you're looking at it from the angle of having a solution and looking for a problem.
SEO is, whether white hat or black hat, a matter of manipulating the results of a search engine to suit a purpose. Any search engine needs to exist before one can figure out how to manipulate it. There isn't much pre-calculating which can be done before you know how a system works, and luckily, even with "internet time", it takes significant time for the behavior of a large portion of the consumer base to shift. If a new search engine "NevDull" came out today, and was considerably better than Google, Yahoo, and MSN, there are market forces which will give any SEO enough time to work with and learn to optimize (manipulate) the results before the hordes of traffic hit.
SEO can't really be ahead of the curve, it can just react quickly.
We don't make the rules. Sometimes we don't even know the rules, only the behaviors. We work with implementations.
Understand what you mean,
and yes, I do have a solution, and a problem, and looking for a good implementation method.
Hype about vitamins?
Not exactly is it. This is pure research, what on earth do you think people do? Look at internet forums and articles and say "ah yes, thats what is happening and what will change".
I don't really talk about search engines, that's because I don't want to. I don't think its a huge issue since there are different ways to extract and study data than that.
I hope you don't go out to "manipulate" the index but rather clean up the sites so they comply better.
randfish
02-22-2005, 04:42 PM
Xan,
Are you suggesting that as grid computing becomes more popular, it will create 'communities' of information on the Internet, and searchers and egnines will target these particular communities, rather than (or as an alternative to) the web as a whole. I could see many implications for SEO - the need to have multiple presences targeting each community, the need to find and focus on the best communities for your subject, etc.
But, I'm not sure whether this is a direction you're suggesting.
Yes Randfish, that's the idea, its pure speculation, but nothing is impossible, at least in computer science!
rustybrick
02-22-2005, 05:05 PM
I am not sure that (randfish's explanation of xan's idea) would be the way to go for grid computing.
I thought that was your idea as well. But the best way, IMO and at this time, is to look at off page factors to determine that. Like Ask does.
Marshall Clark
02-23-2005, 07:32 PM
I agree that this about as close as Sci Fi (and I'm still waiting for my hovercar dammit :)) but there are some interesting possibilities however remote. Some of these ideas are no doubt half-baked, but I make part of my living as an inventor so half-baked ideas are my part and parcel:
1. Google develops a grid application that users install on their machines in return for free processing time on the Google AtHome network. Google uses script to run distributed search/ranking processing and monitor local P2P information distribution.
2. Incorporating P2P file search/transfer information into existing web content. Referral breadcrumbs added to P2P distributed content trace the pulse of dynamic information through the web. Something like the contextual network/spreading activation model of energy/information transfer but using dynamic content and relationships.
3. Large corporations partner with Google to host local search nodes utilizing their huge databases of industry/vendor/customer information. (scary this one – Do No Evil…)
4. Ultra-specialized distributed search nodes placed within contextual networks to give up-to-the-second ranking results on topic specific dynamic information (thinking ~continuous RSS feeds).
NevDull
02-23-2005, 07:52 PM
I agree that this about as close as Sci Fi (and I'm still waiting for my hovercar dammit :)) but there are some interesting possibilities however remote. Some of these ideas are no doubt half-baked, but I make part of my living as an inventor so half-baked ideas are my part and parcel:
1. Google develops a grid application that users install on their machines in return for free processing time on the Google AtHome network. Google uses script to run distributed search/ranking processing and monitor local P2P information distribution.
This is distributed computing, not grid computing... and I'm missing what benefit the customer derives from this. As for "monitoring local P2P information distribution", I don't see how it could be done without sniffing the wire or distributing their own P2P app. I assume you're familiar with the grub.org crawler?
2. Incorporating P2P file search/transfer information into existing web content. Referral breadcrumbs added to P2P distributed content trace the pulse of dynamic information through the web. Something like the contextual network/spreading activation model of energy/information transfer but using dynamic content and relationships.
There would be economic benefit to publishers to use bittorrent instead of Akamai, though service levels are certainly not measurable if you do that.
3. Large corporations partner with Google to host local search nodes utilizing their huge databases of industry/vendor/customer information. (scary this one – Do No Evil…)
You mean like if they bought a Google Appliance and ran it against their own data? :)
"Provide access to data stored in your relational databases by crawling the content directly from the database and serving the content in a number of formats including IBM DB2, Microsoft SQLServer, MySQL, Oracle and Sybase."
Google Search Appliance Features (http://www.google.com/enterprise/gsa/features.html)
4. Ultra-specialized distributed search nodes placed within contextual networks to give up-to-the-second ranking results on topic specific dynamic information (thinking ~continuous RSS feeds).
I don't see where grid computing fits in here, either.
Grid computing is about utility computing. Resources on demand. Flick on a light switch, and it's there. Not simply distributed computing. Not clustering. (hell, Google's been doing some amazing distributed systems work for a while now - I read a great blog entry a few months ago looking at their server farm as more of a platform in itself, with things like gmail and such just laid atop a highly-redundant, self-healing, cheap-parts foundation)
-Anthony
Marshall Clark
02-23-2005, 08:21 PM
Lol Anthony - I think you made it about as far into the future as next week :)
..and I'm missing what benefit the customer derives from this.
Finding motivations and incentives for providing access are what the marketing execs get paid for. Why would people ever allow Google to spy on them? Good question. Why do people use the Google Toolbar?
This is distributed computing, not grid computing...
"Indeed Grid Computing promises to deliver the power of distributed computing to the desktop for years to come"
Sun Powers the Grid: An Overview of Grid Computing From Sun (http://www.sun.com/products/hpc/pdfs/grid.pdf)
You mean like if they bought a Google Appliance and ran it against their own data? :)
Exactly - and then went that crucial extra step of actually dumping the contents on to the web... ;)
Hehehe, I agree Marshall, its definately a far in the future question! Like I said, speculation is healthy, keeps the mind open.
hardball
02-24-2005, 09:52 AM
its definately a far in the future question!
Wimax will bring it all into focus quickly. For most of us sitting in front of a desktop/laptop is comfortable. Wimax will breed a new OS like experience; small, portable devices tapping into "grid" applications and storage (watch a twelve year old with a cell phone to see the future) is the not so distant future. Oddly enough it isn't google who is positioned to be the gateway to the grid, the grid is a closed system with access controlled by the provider, the ISP so to speak, the gbot won't be running loose.
Marshall Clark
02-25-2005, 04:58 PM
the grid is a closed system with access controlled by the provider, the ISP so to speak, the gbot won't be running loose.
Hi hardball,
I'm not sure I agree with this - couldn't the same thing have been said about ISPs under the current Internet framework? AOL made a play at creating their own closed system of content & access and we all know how that turned out.
hardball
02-25-2005, 09:33 PM
Hi Marshall
There has to be a value proposition for the user. If you are going to know my business I want more from you than contextual based ads. You need to deliver genuine value. AOL actually missed the network OS window, they could have had a shot if they put a small OS on all those floppies and CDs that people used to install. They had trust, content and quite a few other goodies like auto-updating, etc.. For all the naysaying about AOL, it is still pretty close to a "darn good" platform. They also answer the phone when you call (I can't see G doing that) ;) .
AOL doesn't let crawlers go willy nilly through chat transcripts to identify "communities" but they do benefit from the user data that they have, they earned it by giving back value.
I saw Terry Semel, Eric Schmidt, and the guys from Intel and Cisco discussing the "grid" and it was painfully obvious google was missing the boat. Yahoo is putting together a robust content delivery system with real live content agreements, its no mistake that Terry Semel has an entertainment background, google is at the mercy of robots.txt.
I'll go camp out on your grid and let you peek into my tent but you better have something besides search and ubiquitous ads for me.
It seems I am not the only one in the online community to be thinking about the future of the grid and the future of the internet and search.
A film maker has made a great short film about the past and future of the internet.
Watch it and see what you think. Either way, it's beautifully done.
EPIC 2014 (http://mccd.udc.es/orihuela/epic/)
(Evolving Personalization information construct)