What is the difference between crawling and indexing?

Status
Not open for further replies.

chrishirst

Well-Known Member
Staff member
Crawling is the retrieval of the source code from a URL and storing the data in the 'indexer' queue database.

Indexing is the process of reading the stored source data and adding the words, along with their positions in the document to an 'inverted index' and associating that data with the document ID for later scoring and weighting.
 

jainteq

Member
Crawling: When Google visits your website for tracking purposes. This process is done by Google’s Spider crawler. Indexing: After crawling has been done, the results get put onto Google’s index (i.e. web search).
 

chrishirst

Well-Known Member
Staff member
When Google visits your website for tracking purposes.

"Tracking" has nothing to do with it, unless you are using 'tracking' in whole other way than the usual.

So:

if you mean 'tracking' as in "checking the URL for changes to the content" ... ... No it isn't.

if you mean 'tracking' as in "checking the URL following some 'ping' request" ... ... No it isn't.

if you mean 'tracking' as in "looking at some visitor statistics" ... ... No it isn't.

if you mean 'tracking' as in "checking the URL after a new 'link' is discovered" ... ... Not really but that may happen.


An individual URL, NOT a 'website' is crawled on a schedule determined by Google's system, seemingly based on links and a Google maintained history of when the content at the particular URL has been changed when previously visited.
 

hariandro95

New Member
hello phpassignmenthelp , glad to answer this question because i also had this doubt before.Lets know about that..
Crawling
Crawling is the process search engines discover updated content on the web, such as new sites or pages, changes to existing sites, and dead links.
To do this, a search engine uses a algorihthat can be referred to as a 'crawler', ‘bot’ or ‘spider’ which follows an algorithmic process to determine which sites to crawl and how often.As a search engine's crawler moves through your site it will also detect and record any links it finds on these pages and add them to a list that will be crawled later. This is how new content is discovered.

Indexing
Once a search engine processes each of the pages it crawls, it compiles a massive index of all the words it sees and their location on each page. It is essentially a database of billions of web pages.
This extracted content is then stored, with the information then organised and interpreted by the search engine’s algorithm to measure its importance compared to similar pages.
 

Steve robert

New Member
Crawling means google has crawled your site atleast once and your site goes into their crawled database. Indexing means google has crawled your site and found it valuable enough to put it in their indexed pages database.


link dropping removed
 
Last edited by a moderator:

chrishirst

Well-Known Member
Staff member
Crawling means google has crawled your site atleast once
That is a circular 'definition', you cannot use two verbs derived from the same word to 'explain' the original word.

Besides that you are incorrect , because crawling is a present tense verb while, crawled is a past tense verb. So crawling is performing the action of a 'crawl' while crawled is after the 'crawl' is completed.


Indexing means google has crawled your site and found it valuable enough to put it in their indexed pages database.
No again, indexing is purely the operation of reading the retrieved data when the source URL was crawled at this point all indexed URLs are added to their database 'valuable' or not
 

chrishirst

Well-Known Member
Staff member
Crawling implies the web index robot creep or get the site pages while Indexing implies web search tool robot slithers the pages, spared the information and it shows up in the internet searcher.
THAT could well be the very WORST example of automatic "translation" that I have EVER seen, and the article 'spinner' you obviously used then turned it into complete gibberish.


The word is crawl, not creep or slithers, neither of those can be a synonym for 'crawl' when writing or speaking in a search engine context
 
Status
Not open for further replies.
Top