Search Engine Optimization
Would you like to react to this message? Create an account in a few clicks or log in to continue.
Search Engine Optimization

open discussion

Affiliates
Use of robot.txt ? Image-6549946-10459049
Latest topics
» How do you insert images in HTML?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:39 pm by Designpluz

» What tag is used to insert the main and largest heading in HTML?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:39 pm by Designpluz

» Is any of the following software suitable for building websites?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:38 pm by Designpluz

» Browsers apply the tag feature until they encounter a ___ tag.
Use of robot.txt ? EmptyTue Feb 14, 2023 1:38 pm by Designpluz

» Which language is helpful to create web pages?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:38 pm by Designpluz

» Which is the most common language used in web designing?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:37 pm by Designpluz

» Can comments be inserted into HTML code without being picked up by the browser?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:37 pm by Designpluz

» What is the point of marking an element as an ARTICLE, MAIN, HEADER, or ASIDE?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:36 pm by Designpluz

» Are there any ways to reduce the loading time for a page?
Use of robot.txt ? EmptyTue Feb 14, 2023 1:36 pm by Designpluz


Use of robot.txt ?

5 posters

Search Engine Optimization :: Search Engine Corner :: Search Engine Optimization

Go down

Use of robot.txt ? Empty Use of robot.txt ?

Post by zinavo Tue Apr 11, 2017 7:40 pm

Use of robot.txt ?

zinavo
platinum
platinum

Posts : 105
Earned : 318
Credit : 3
Join date : 2016-10-11

Back to top Go down

Use of robot.txt ? Empty Re: Use of robot.txt ?

Post by zorbee Wed Apr 19, 2017 3:08 am

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

zorbee
platinum
platinum

Posts : 131
Earned : 271
Credit : 0
Join date : 2016-08-31
Age : 35
Location : Delhi

Back to top Go down

Use of robot.txt ? Empty Re: Use of robot.txt ?

Post by bangalorewebguru Wed May 03, 2017 9:08 pm

Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.

bangalorewebguru
platinum
platinum

Posts : 1204
Earned : 4896
Credit : 4
Join date : 2013-10-15
Age : 31
Location : Bangalore

Back to top Go down

Use of robot.txt ? Empty Re: Use of robot.txt ?

Post by janpaul7890 Thu May 04, 2017 8:44 pm

Robots.txt is a simple text file webmaster creates to instruct search engine crawler how to crawl and indexed of their web pages.

janpaul7890
platinum
platinum

Posts : 72
Earned : 144
Credit : 0
Join date : 2016-08-10

Back to top Go down

Use of robot.txt ? Empty Re: Use of robot.txt ?

Post by hariandro Thu May 25, 2017 5:54 pm

Robots.txt - This text file which sits in the root of your website's folder communicates a certain number of guidelines to search engine crawlers. For instance, if your robots.txt file has this line in it; User-agent: * Disallow: / it's basically telling every crawler on the web to take a hike and not index ANY of your site's content.

hariandro
platinum
platinum

Posts : 145
Earned : 371
Credit : 0
Join date : 2016-12-22
Location : chennai

http://www.samsmarine.org

Back to top Go down

Use of robot.txt ? Empty Re: Use of robot.txt ?

Post by Sponsored content


Sponsored content


Back to top Go down

Back to top

Search Engine Optimization :: Search Engine Corner :: Search Engine Optimization

 
Permissions in this forum:
You cannot reply to topics in this forum