Knowledgebase

How to block search bots from accessing your site?  Print this Article

In order to block all bots from accessing your site, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /

To block all bots from accessing a specific folder, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /folder/

Please note that blocking all bots will get your site deindexed from search engines.

Was this answer helpful?

Related Articles

Error Page Codes
These numbered error codes are how the server tells website visitors about an error encountered....
What do you mean when you say gigabyte, megabyte, GB, and MB?
The smallest unit of measurement on a computer is the bit. There are almost always eight bits in...
Propagation Times
Propagation is the process of updating a domain with new nameserver (DNS) information across all...
What is an Uptime Guarantee?
Server uptime refers to the amount of time in a given period a server stays up and running....
Why shouldn't I go for that unlimited plan? Beware overselling.
If you've been searching for a web hosting provider, you have probably seen some offers that look...