Home > Articles > Operating Systems

  • Print
  • + Share This
This chapter is from the book

This chapter is from the book

SEARCH THE INVISIBLE WEB

By Tom Merritt and Martin Sargent

Huge portions of the Internet are invisible to most search engines.

Some folks call this the invisible Web. Don't confuse the invisible Web with HTML pages that don't get indexed by the major engines. You can still find these pages by using specialty search engines sucha Artcyclopedia.com and Lawcrawler.com. We include a list of these at the end of this feature.

Search engines rely on programs called spiders, which only index HTML pages. Spiders can't index CGI scripts, PDF files, or information in databases. Various sources estimate you can find about 500 times more information in databases than in normal search engines.

DATABASE DIRECTORIES

These sites gather together online databases. You can't actually search the databases, but you can pinpoint just the databases you need:

  • Direct Search (gwis2.circ.gwu.edu/~gprice/direct.htm), compiled by Gary Price of George Washington University, is a great resource for databases and for more information on the invisible Web.

  • InvisibleWeb.com (www.invisibleweb.com) was created by Intelliseek, maker of BullsEye search software and ProFusion.

DATABASE SEARCH ENGINE

ProFusion (www.profusion.com) from Intelliseek searches more than 1,000 information sources, including the invisible Web. It works by formatting your search criteria to meet the specific requirements of each data source. Check out its features:

  • Automatically searches across multiple sources and categorizes the results.

  • Suggests alternative searches based on information it has learned.

  • Allows searches by category, or by source within a category.

  • Enables you to automate searches and enables alerts when search results change.

  • Does not require specialized search skills.

SPECIALTY SEARCH ENGINES

Yahoo! and Google aren't always the best tools for finding information on a particular topic. Here's a partial list of some excellent specialty search engines that can help you find exactly what you're looking for.

Art

Artcyclopedia (www.artcyclopedia.com) is an amazing site. Artcyclopedia offers a database of works by 7,500 artists from the world's art museums and image archives.

Aviation

FlightSearch.com (www.flightsearch.com) has everything from aviation pictures and products to information on flight disasters.

Computer Games

There are several decent computer gaming directories on the Web, but these are the most helpful:

Computer Security

Go to SecureRoot (www.secureroot.com) and search more than 15,000 URLs for information about hacking, cracking, encryption, and anarchy.

Computer Programming

Are you a software developer? Want to learn more about programming? SourceBank (www.devx.com/sourcebank) has the goods.

Employment

Find postings from the Web's many job-listing sites on FlipDog (www.flipdog.com).

Fishing

Find everything from learning techniques to buying tackle on FishSearch.com (www.fishsearch.com).

Law

Trouble with the fuzz? Find legal information using these legal-oriented search engines:

Medicine

Feeling some symptoms? You might know about WebMD, but to broaden your search try a medical search engine. These specialty search engines find documents about particular conditions and medical issues:

Money

FinancialFind.com (www.financialfind.com) provides a comprehensive directory of financial information on the Internet.

Museums

MuseumStuff (www.museumstuff.com) has all you need to know about thousands of museums worldwide, nicely parceled into categories.

News

Moreover (www.moreover.com) is perhaps the best specialty search engine in any category. Moreover serves up headlines from more than 1,800 sources. Most important, the headlines retrieved are up to date.

Politics

OneWorld.net (www.oneworld.net) offers information on human rights and environmental issues worldwide.

Uncle Sam (www.google.com/unclesam) is your source for everything .gov. It's still the same Google you've grown to love, but results are limited to U.S. government Web pages.

Shopping

There are scores of comparison-shopping sites, but MySimon.com (www.mysimon.com) is the best. It finds the best price from some 1,600 online merchants.

Software

If you want downloadable software, try these sites:

Search engines such as Google can't access much of the information hidden in documents and databases. To find this information, use a search engine that specializes in this Deep Web.

Try InvisibleWeb.com (www.invisibleweb.com) and Direct-Search (www.directsearch.com). They'll pull useful information out of databases for you.

You might want to try a service by BrightPlanet called LexiBot (www.lexibot.com). BrightPlanet claims that LexiBot can get at the roughly 550 billion documents in the Deep Web. According to BrightPlanet, regular search engines can find only 1 billion documents.

With 500 times more information to retrieve, LexiBot isn't too easy to master. The first time you take it for a test spin you might be confused. That's okay, because you have 30 days to play with it before you have to shell out $90.

Tom Merritt is Executive Web Producer at TechTV. Martin Sargent is a writer and guest host at TechTV.

  • + Share This
  • 🔖 Save To Your Account