How to block search bots from accessing your site?

In order to block all bots from accessing your site, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /

To block all bots from accessing a specific folder, you should create a robots.txt file with the following content:

User-agent: *
Disallow: /folder/

Please note that blocking all bots will get your site deindexed from search engines.

  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

How do I disable directory browsing?

By default, if you don't have an index file in a web accessible folder, when accessed through a...

Should I be taking backups of my account? If so, how often?

Backups are crucial to the smooth operation of any website, no matter how important, how many...

What do you mean when you say gigabyte, megabyte, GB, and MB?

The smallest unit of measurement on a computer is the bit. There are almost always eight bits in...

What is a MySQL database?

A MySQL database, like all other types of databases, is used to keep track of a website or...

Can people find out where I live based on my domain registration information?

Unfortunately, the answer is usually going to be yes. Any address and contact information you...