Magento Expert Forum - Improve your Magento experience

Results 1 to 7 of 7

What is a robots.txt file?

  1. #1

  2. #2
    Junior Member
    Join Date
    Feb 2016
    Posts
    157
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Default

    Robots.txt is a text file that is inserted into your website and contains information for search engine robots. The file lists webpages that are allowed and disallowed from search engine crawling.

  3. #3
    Junior Member
    Join Date
    Apr 2016
    Location
    Delhi
    Posts
    117
    Thanks
    1
    Thanked 1 Time in 1 Post

    Default

    Quote Originally Posted by orientation View Post
    hi friends,

    What is a robots.txt file?
    Robots.txt is txt file. which helps to handle the pages, we can tells us to google which pages index or not. read more about robots.txt read this https://support.google.com/webmaster.../6062608?hl=en

  4. #4
    Junior Member
    Join Date
    Mar 2016
    Posts
    220
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Default

    Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.

  5. #5

  6. #6
    Junior Member
    Join Date
    May 2016
    Posts
    30
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    A robots.txt file is a digital "Keep Out" sign, designed to keep web crawlers out of certain parts of a web site.

  7. #7
    Junior Member
    Join Date
    May 2016
    Location
    Punjab India
    Posts
    137
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default

    It is a essential for a website. It contains instructions for crawlers.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •