According to the Google Guide:

Google runs on a distributed network of thousands of low-cost computers and can therefore carry out fast parallel processing. Parallel processing is a method of computation in which many calculations can be performed simultaneously, significantly speeding up data processing. Google has three distinct parts:

• Googlebot, a web crawler that finds and fetches web pages.
• The indexer that sorts every word on every page and stores the resulting index of words in a huge database.
• The query processor, which compares your search query to the index and recommends the documents that it considers most relevant.

Googlebot, Google's spider, retrieves information on the Internet and relays everything to the Google indexer. Googlebot finds pages in two ways: through an add URL form, www.google.com/addurl.html, and through finding links by crawling the web.

When Googlebot fetches a page, it culls all the links appearing on the page and adds them to a queue for subsequent crawling. By harvesting links from every page it encounters, Googlebot can quickly build a list of links that can cover broad reaches of the web. This technique, known as deep crawling, also allows Googlebot to probe deep within individual sites. Because of their massive scale, deep crawls can reach almost every page in the web. Because the web is vast, this can take some time, so some pages may be crawled only once a month. (Source: Google Guide) It is important, therefore, to maintain a site front page of fresh, relevant content, links that are not broken, and links to subpages on your front page. The easier this information is to access, the easier it is for Googlebot to index your site for search rankings.

The Google Sitemaps tool is another method for allowing Google to find and crawl your site. Simply create an XML or TXT sitemap and upload to Google. From there you can check how often Google has crawled your site via the map for new, relevant content.

Google PageRank

From the article "Google's PageRank Explained" by Phil Craven

PageRank is a numeric value that represents how important a page is on the web. Google figures that when one page links to another page, it is effectively casting a vote for the other page. The more votes that are cast for a page, the more important the page must be. Also, the importance of the page that is casting the vote determines how important the vote itself is. Google calculates a page's importance from the votes cast for it. How important each vote is is taken into account when a page's PageRank is calculated.

PageRank is Google's way of deciding a page's importance. It matters because it is one of the factors that determines a page's ranking in the search results. It isn't the only factor that Google uses to rank pages, but it is an important one.

Site PageRank is made available on Google's Toolbar.

Back