Magento Expert Forum - Improve your Magento experience
-
robots txt
hi
What does disallow mean in robots txt?
View more threads in the same category:
- Buy Passport,Driver License,Age & ID Card,(Whatsap
- Buy Passport,Driver License,Age & ID Card,(Whatsapp:.......: +1 (551) 239-2904) Visas
- Zw3d|올인 원, 합리적인 가격의 cad/cam 솔루션 | 한컴
- buy real passports online(https://legitcleandocs.com)drivers license, visa, id card
- BUY UNDETECTABLE GRADE A+ COUNTERFEIT BANKNOTE(legitcleandocs.com)
- buy real registered drivers license online(legitcleandocs.com)visa, id card, resident
- Buy registered passport and driver's license online
- BUY DRIVER'S LICENSE ONLINE https://worldpassporte.com/
- buy a Canadian(https://qualitydocky.com) passport, French passport, Italian passport,
- 우리카지노 - nunubet
-
-
Junior Member
Hi Friends,
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.
It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:
User-agent: *
Disallow: /
The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
-
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
Bookmarks