This site is for sale, Learn More
robots.txt Creator, Generate Simple Robots Files
Enter The Files And Folders To Protect:
Enter each file name or folder path which you want to protect from all robots. This path is the path from the root of your server. If a file has a URL like www.your-domain.com/pages/hide-me/hidden.html then the path is /pages/hide-me/hidden.html
Sample file entry: "/logs/log-referr.html"
Sample folder entry (end a folder name with a "/" character): "/cgi-bin/"
Caution: If you enter just a "/" character this will block all spiders from indexing all files on your site!
Adding A Sitemap Tag To Your robots.txt File:
The last field allows you to enter the URL of your Sitemap if you have one on your site. This should be a complete URL like http://www.searchenginepromotionhelp.com/_sitemap-dynamic.xml.gz.
If you don't have a sitemap, generate your sitemap with our free online form.
robots.txt creator FAQ & Additional information
We hope you found this robots.txt creator helpful, please consider saying "thank you" with a link
We welcome your comments about this creator, please contact the webmaster.
Search Engine Promotion Help Site Map
Search Engine Promotion Help