How to block search bots from accessing your site?

In order to block all bots from accessing your site, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /

To block all bots from accessing a specific folder, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /folder/

Please note that blocking all bots will get your site deindexed from search engines.

  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

Error Page Codes

These numbered error codes are how the server tells website visitors about an error encountered....

What do you mean when you say gigabyte, megabyte, GB, and MB?

The smallest unit of measurement on a computer is the bit. There are almost always eight bits in...

Propagation Times

Propagation is the process of updating a domain with new nameserver (DNS) information across all...

What is an Uptime Guarantee?

Server uptime refers to the amount of time in a given period a server stays up and running....

Why shouldn't I go for that unlimited plan? Beware overselling.

If you've been searching for a web hosting provider, you have probably seen some offers that look...