What is robot.txt file?

View Latest Questions and Answers|Ask Questions?|Tutorials

Home Answers Viewqa HTML What is robot.txt file?

Login

QuestionAsk Questions?

Deepak Kumar

What is robot.txt file?
1 Answer(s) 2 years and 4 months ago
Posted in : HTML

Hi,

What is robot.txt file?

Thanks

View Answers

January 26, 2011 at 12:36 AM

Hi,

The robot.txt file is places in the root of the website.

It allows the web masters to instruct the website pages for indexing in the search engine.

You can also disallow the search engine for indexing or crawling certain url's of your website.

Here is example of robot.txt file:

User-agent: *
Allow: /

Thanks

Related Pages:

What is robot.txt file?

What is the purpose of the following files having extensions: frm, myd, and myi? What these files contain?

what are the codes for uploading and downloading files in and from database?

Concatenate two pdf files

configuration files are used in Struts

library files - Development process

configuration files - Java Beginners

Reading files in Java

Dynamic include of multiple files

files

files

Concatenate pdf files using command argument

What is CVS

files

files

files

files

files

Files

Hibernate required jar files - Hibernate

What is a package?

Reading Files and Print - Java Beginners

creating index for xml files - XML

What is JDBC?

what are the jar files should i add to resolve the Exception(Class org.apache.jsp.index_jsp Not Found) ?

Necessary Jar and War files for struts - Struts

About tld files - JSP-Interview Questions

combine two pdf files

if we have 2 struts-config files - Struts

Uploading multiple files in JSP - JSP-Servlet

JSP Upload and Downloading files - JSP-Servlet

Merging multiple PDF files - Framework

Compare 2 files

what is the resolution for the bellow error?

what is xfire? - Java Beginners

Self-reloading XML Property Files - Java Tutorials

Self-reloading XML Property Files - Java Tutorials

What is difference between Path and Classpath?

What is AAC?

What is a Java package and how is it used?

header files

what is the default buffer size for bufferedreader

What is the difference between a JDK and a JVM?

What is a database?

Get Current Directory

Java Collections - Design concepts & design patterns

Technology What is and FAQs

If I open .class file with notepad what is displayed?

validation files

Java File - Learn how to handle files in Java with Examples and Tutorials

If you are facing any programming issue, such as compilation errors or not able to find the code you are looking for.

Ask your questions, our development team will try to give answers to your questions.

Tutorial Topics