Robots.txt Generator
About this tool
Build and download a robots.txt starting point, then review it before publishing.
Privacy
Your input is processed locally in your browser and is not uploaded by this tool.
Frequently asked questions
Does robots.txt prevent indexing?
Not reliably by itself. Use page-level noindex controls when indexing must be prevented.
Where should robots.txt be placed?
It belongs at the root of the hostname, such as example.com/robots.txt.
Should private pages rely on robots.txt?
No. Protect private content with authentication and proper access controls.