Google’s

rolled out a new tool at Google
Webmaster Central, a robots.txt generator. It’s designed to allow site
owners to easily create a robots.txt file, one of the two main ways (along with
the meta robots tag)
to prevent search engines from indexing content. Robots.txt generators aren’t
new. You can find many of them out there by searching. But this is the first
time a major search engine has provided a generator tool of its own.

It’s nice to see the addition. Robots.txt files aren’t complicated to create.
You can write them using a text editor such as notepad with just a few simple
commands. But they can still be scary or hard for some site owners to
contemplate.

Click to continue reading…

Read Original Post Here