Website & SEO tools

Domain, SEO, security, URLs, text, images, dev utilities—each tool has its own URL.

Robots.txt Generator

Robots.txt generator

Download robots.txt

Guide: Robots.txt Generator

Topics in this category: On-page SEO and metadata

Robots.txt role in SEO

Robots.txt is a crawl directive file at your site root. It tells compliant bots which paths they should not fetch—it does not enforce security or hide private URLs from determined actors.

Do not block CSS, JS, or assets required to render pages if you want Google to understand layout and content. Include your sitemap URL as a hint for discovery.