As the amount of information on the Internet continues to grow, A
startup called 80legs, based in Houston, TX, is hoping that an
inexpensive, distributed Web crawling service could help startups mine the Web
for information without having to build the giant server farms used by major
search engines. Web crawlers, or spiders, are software that automatically visit
pages on the Internet and can be used to index them and gather bits of
information from different pages.
