Magento Expert Forum - Improve your Magento experience

Results 1 to 4 of 4

web crawler

  1. #1

  2. #2
    Junior Member kajal's Avatar
    Join Date
    Sep 2014
    Location
    Bangalore
    Posts
    1,446
    Thanks
    0
    Thanked 20 Times in 20 Posts

    Default

    Hi Friends,
    Googlebot is the name of Google's web crawler. A web crawler is an automated program that systematically browses the Internet for new web pages. This is called web-indexing or web-spidering. Google and other search engines use web crawlers to update their search indexes.

  3. #3
    Junior Member
    Join Date
    Feb 2015
    Posts
    316
    Thanks
    0
    Thanked 3 Times in 3 Posts

    Default

    A Web crawler is an Internet bot which helps in Web indexing. They crawl one page at a time through a website until all pages have been indexed. Web crawlers help in collecting information about a website and the links related to them, and also help in validating the HTML code and hyperlinks.

  4. #4
    Junior Member
    Join Date
    Jul 2018
    Posts
    563
    Thanks
    6
    Thanked 3 Times in 3 Posts

    Default

    Googlebot is the name of Google's web crawler. A web crawler is an automated program that systematically browses the Internet for new web pages. This is called web-indexing or web-spidering. Google and other search engines use web crawlers to update their search indexes.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •