COMMUNIUM.AI

danielmiessler/RobotsDisallowed

A curated list of the most common and most interesting robots.txt disallowed directories.

1,481
Shell
View on GitHub
Stars

1,481

Updated

Nov 24, 2025

Stars Over Time

Top Contributors

KU

kurosakilabs

1 contributions

MA

marwan-cpu

1 contributions

ZK

Zko666

1 contributions

AR

arsend

1 contributions

JH

jhartzel

1 contributions

AW

awaliuddin

1 contributions

SH

Shasheen8

1 contributions

CH

ChrisHanlonCA

1 contributions

JA

jake-fleming

1 contributions

AH

ahmetburakakay

1 contributions

4E

4evsec

1 contributions

NO

noods78

1 contributions

TU

tubaking

1 contributions

TO

tommoholmes

1 contributions

RA

Ramyromel

1 contributions

MI

migozarad

1 contributions

VI

Vict0rFrankenst31n

1 contributions

KE

keyboardbash

1 contributions

RI

richknowles

1 contributions

MO

Mohammad-Almusilhi

1 contributions

KA

karamimoheb

1 contributions

VI

Vianndev

1 contributions

HO

hollisticated-horse

1 contributions

3L

3ls3if

1 contributions

MA

markosorec

1 contributions

LE

Leak1ll

1 contributions

KU

Kunhunni

1 contributions

MA

Mariven

1 contributions

JA

jasoneterry

1 contributions

99

99red

1 contributions

PL

plashal

1 contributions

ID

idjosep

1 contributions

ME

melnikovics

1 contributions

BF

bfernald

1 contributions

JO

jon91

1 contributions

DJ

djd0723

1 contributions

RE

reapbooster

1 contributions

ST

stochasticity

1 contributions

LE

leana8959

1 contributions

SH

shayansec

1 contributions

OW

owais-zeron

1 contributions

GO

goldfishlaser

1 contributions

D4

d43ash1sh

1 contributions

RU

RustyCobalt36

1 contributions

SE

seanclancy

1 contributions

LE

levonk

1 contributions

AI

AiorcaLin

1 contributions

CU

curthayman

1 contributions

DA

DANIEL-lang-sudo

1 contributions

LU

luthiena

1 contributions

SI

sinkafish

1 contributions

CY

CyberAlbSecOP

1 contributions

MI

miiiladiii244

1 contributions

IN

InfraredReactivePACN

1 contributions

XT

xthemadgenius

1 contributions

TU

tuantuansun

1 contributions

CB

cbyt3

1 contributions

OL

ol39n1

1 contributions

M4

m4ra7h0n

1 contributions

IS

Isae-LM

1 contributions

BR

Britz-rabbit

1 contributions

TE

Technetium1

1 contributions

LO

Loweredgames

1 contributions

XA

xajkep

1 contributions

JA

JackChiang0130

1 contributions

PU

purpole

1 contributions

LA

lab-modular

1 contributions

4K

4kray

1 contributions

RH

rhdhtmrm

1 contributions

AL

alinescu16

1 contributions

JO

johnny5412

1 contributions

WA

wafflexyzz

1 contributions

YI

yida223

1 contributions

ID

idriskhenchil

1 contributions

RO

Rokkrs

1 contributions

SO

Southrngurl

1 contributions

KA

kashifraza543

1 contributions

LR

LRose7

1 contributions

BY

Byrnesz

1 contributions

CR

CRYPTOJEFFE

1 contributions

JR

jrubio-bf

1 contributions

RY

ryanklsue

1 contributions

CX

cxzero

1 contributions

RE

REALSDEALS

1 contributions

4X

4x45c0

1 contributions

HA

hastefuI

1 contributions

SO

sokai

1 contributions

ET

ethicalplasma

1 contributions

SA

safenest-cyber

1 contributions

DE

demondarkspace

1 contributions

PR

PR0F0X01

1 contributions

MI

miethe

1 contributions

ES

ESDCDimitriViel

1 contributions

AM

amad3i

1 contributions

CA

canering

1 contributions

T4

t411m4dg3

1 contributions

WH

whoIsStella

1 contributions

ST

stlcours

1 contributions

C4

c4kar

1 contributions

CE

ceaserone

1 contributions

Related Repositories

wireghoul/graudit

1,662 stars

ashishb/osx-and-ios-security-awesome

1,537 stars

neologd/mecab-ipadic-neologd

2,789 stars

Track developers from danielmiessler/RobotsDisallowed

Join 1,000+ companies finding quality developer leads