Search is the fighting arena for Yahoo, Microsoft and Goggle and who ever is dominating in Internet searches is the master of the Internet.
Recently Microsoft introduced Search Together (collaborative search on the net) , and now Yahoo is trying something even more daring – opening its search technology for web developers.
With BOSS ( Build your Own Search Service ) " developers, start-ups, and large Internet companies can use BOSS to build and launch web-scale search products that utilize the entire Yahoo! Search index. BOSS gives you access to Yahoo!'s investments in crawling and indexing, ranking and relevancy algorithms, and powerful infrastructure." More details are available from TechCrunch.
Google also explained recently how they master Internet searches behind the scenes.
"For example when a user in San Francisco enters a query like google.com/search?q=blogoscoped, the user’s browser first completes a DNS lookup mapping http://www.google.com to a specific IP address. At this stage, Google’s DNS load balancer determines which cluster of computers at which of Google’s 36+ data centers will process the query."
Once the search query arrives at the correct data center, another load balancer determines which one of the hundred servers will process the request. Then the search goes to a Google mixer, which strips the question into smaller info requests and sends them to the servers, and later assembles all finished information pieces and posts them back to the data center.
Google Blogoscoped has more info, if you want to find out more about how Google handles searches.