Magento Expert Forum - Improve your Magento experience

Results 1 to 4 of 4

How to Block folder and file through Roots.txt File?

  1. #1

  2. #2

  3. #3
    Junior Member
    Join Date
    Jul 2018
    Posts
    625
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Disallow folder robots txt

    Below are some helpful commands that can used to Disallow files and folders on your website in your robots.txt file.
    Block your entire website from crawling: User-agent: * Disallow: /
    Block particular files from crawling: User-agent: * Disallow: /example1.html. ...
    Block particular folders from crawling: User-agent:

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •