Sunday, December 21, 2008

crawlers/ robots/ Googlebot

crawlers/ robots/ Googlebot - Also known as a "bot" or a "spider", a search engine spider follows links to web pages and then reads and retains the information it finds. This information eventually becomes the "copy" of a website in a search engine index. This process is often referred to as "crawling" the web. "Googlebot" is the name of the search engine crawler that is most used by Google.

No comments:

Post a Comment