Parachute Cover
The official online job search resource hosted By Dick Bolles, author of "What Color is Your Parachute"
Research on the Internet | Search Engines
Research on the Internet
 
 
Search Engines
More info:
How Search Engines Work
The Internet is like a huge library with no card index and no titles on the book spines. You can't ever know everything that is there, at any given moment. And so --- imperfect though it may be --- the best tool we currently have for finding specific information on the Internet is the Search Engine. To use search engines effectively, it is important that you understand how they work --- and why sometimes, the data you receive has been paid to be put in front of you, regardless of its relevance. To that end, I have a few links for you to investigate:
Web Searching Tutorial
This is one of the best tutorials on Web Search engines that I have found. Easy to understand, yet very complete.

University of South Carolina - Basic Search Tips
A quick intro on how to formulate search queries. There are links to finding more in-depth information, if you like.

Copyright©1996-2013 by Richard N. Bolles
All rights reserved. No part of this site may be quoted or reproduced without written permission.
For any suggested additions, updating or corrections to this site, please e-mail the Webmaster.
More On How Search Engines Work
close
There are many Search engines on the Net. Some are better than others; and in certain cases, much better. To understand why, and pick the best engine(s) for your research, you need to look at how search engines do what they do.

It's a gross oversimplification, but we could compare a Search engine to the "Find" command in your word processor. When you want to find a certain part of a document, you enter a keyword into the "Find" command, and the word processor software looks for a match.

That's a rough parallel to the way search engines work. In their case, you could say that the Internet is the document; but it's actually more complex than that, and it is in this complexity that we find the differences between the many Search Engines available.

Search Engine differences can be broken down into three factors:

  • The way they index the Web
  • The way that they search the index, return results and rank them by how closely the engine thinks they match your query - the "relevance"
  • The way they deal with advertising and "sponsored matches"
These three factors affect how good the results are that the search engine will return to you when you make a search query. Let's look at each more closely:

The Way They Index The Web: Search engines are computer programs that prowl the Internet (using a software tool called a "crawler" or "spider"), and create a database of pointers to various Web pages (millions of them). Then, when a researcher - say, you, for instance - comes along and says, "Find me data on subject X," the search engine kicks out addresses from its indexed database, from which it has located information it thinks is relevant to your request.

The algorithms the different Search engines use to index the Web can vary significantly from one search engine to another. This means that the resultant indices to the data on the Internet that they each build can also vary significantly. And that affects how they respond to your search queries.

The Way That They Search The Index: Search engines do not search the Internet directly; they search the database they have compiled about the Internet, using their spiders. So, even if two search engines are using the same search technology (very common), the databases that they are using to find your search query results are not the same. (In other cases, different engines use different technology to search the same databases, again yielding different results.) Nor are these algorithms equal in effectiveness, and this means that some of the Search engines are more effective at finding your data than other Search engines. There is overlap, of course; but the overlap is far from complete. This means that when you are conducting an information search, you should not depend on a single engine to give you all of the information that is available out there.

The Way They Deal With Advertising And "Sponsored Matches": All of the Search engines are there to make money. Some will return, along with your search results (but clearly separate), a list of short ads from companies that offer products and services that are usually somewhat related to your search results; it's a kind of targeted advertising. (Google and Yahoo! use this approach.) Other Search engines return results to your search queries that are actually from Web sites that have paid to be returned as search results (often called "sponsored matches"), but are clearly marked as sponsored; Alta Vista and Overture are examples. Still others mix in sites that have paid to be included into your search results, without telling you which is which, regardless of whether those sites rank high in relevance to your search query. According to the U.C. Berkeley Library, MetaCrawler does this, as do others; I don't personally find this ethical, but there it is, and you should be aware of it. Nothing will skew your search for data as badly as having sponsored results mixed in with what I term "relevant results", if you don't know which is which.

close window
close