GitHub repository
danielmiessler/RobotsDisallowed
A curated list of the most common and most interesting robots.txt disallowed directories.
Repository metrics
- Stars · Nov 15, 2026
- 1,484
- Leaderboard status
- Outside the current top 1,000
Language Shell
Source: GitHubLB snapshots.