Latest Tutorials| Questions and Answers|Ask Questions?|Site Map



Home Answers Viewqa HTML What is robot.txt file?
Login         

View Questions and Answers by Category

Advertisements


 
Have Programming Question? Ask it here!
 
 
 


Deepak Kumar
What is robot.txt file?
1 Answer(s)      3 years and 10 months ago
Posted in : HTML


Hi,

What is robot.txt file?

Thanks


Advertisement
View Answers

January 26, 2011 at 12:36 AM


Hi,

The robot.txt file is places in the root of the website.

It allows the web masters to instruct the website pages for indexing in the search engine.

You can also disallow the search engine for indexing or crawling certain url's of your website.

Here is example of robot.txt file:

User-agent: *
Allow: /

Thanks



Related Tutorials/Questions & Answers:
Advertisements
 

 

 

DMCA.com