hosting,  where multiple domain name addresses are mapped to a single IP address.  If

the hosting company were using this method, both www.baseball.com and

www.XXX.com could map to a single IP address, e.g.,  10.3.5.9.   As a result of the

 name based hosting  process, up to tens of thousands of pages with heterogeneous

content may share a single IP address. 

2.  The Indexable Web, the  Deep Web ; Their Size and Rates of Growth and

Change

The universe of content on the Web that could be indexed, in theory, by standard

search engines is known as the  publicly indexable Web.   The publicly indexable Web is

limited to those pages that are accessible by following a link from another Web page that

is recognized by a search engine.  This limitation exists because online indexing

techniques used by popular search engines and directories such as Yahoo, Lycos and

AltaVista, are based on  spidering  technology, which finds sites to index by following

links from site to site in a continuous search for new content.  If a Web page or site is not

linked by others, then spidering will not discover that page or site.  

Furthermore, many larger Web sites contain instructions, through software, that

prevent spiders from investigating that site, and therefore the contents of such sites also

cannot be indexed using spidering technology.  Because of the vast size and decentralized

structure of the Web, no search engine or directory indexes all of the content on the

publicly indexable Web.  We credit current estimates that no more than 50% of the

content currently on the publicly indexable Web has been indexed by all search engines

31