Robots.txt Tester
Test robots.txt against URL.
Test robots.txt rules instantly. Free tester for Allow/Disallow against URL.
Calculator Inputs
About Robots Tester
Test robots.txt against URL.
Checks if URL is blocked by robots rules.
Where it is used
Verify blocking.
How to use the Robots Tester
- Robots.txt Content — enter the value (e.g. User-agent: * Disallow: /private/).
- Test URL Path — enter the value (e.g. /private/page.html).
- Calculate — press the Calculate button to see the result instantly above.
Formula
Last matching Allow/Disallow wins.
Examples
Example
robots:User-agent: *
Disallow: /private/
url:/private/page.html
Result: Blocked
Frequently Asked Questions
- Crawl vs index?
- Robots blocks crawl, not index; use noindex for no-index.