COMMUNIUM.AI

scrapy/dirbot

Scrapy project to scrape public web directories (educational) [DEPRECATED]

1,630
Python
View on GitHub
Stars

1,630

Updated

Nov 4, 2025

Stars Over Time

Top Contributors

A5

a5978445

1 contributions

ZI

zilaijuand

1 contributions

WI

winkerwinker

1 contributions

MS

MSunFlower1014

1 contributions

JA

jasonyun

1 contributions

PA

Pandimurugan12

1 contributions

LI

liuqinyi

1 contributions

DM

dmyers

1 contributions

IM

imjade

1 contributions

WA

waoywssy

1 contributions

CH

Chen-xXing

1 contributions

VE

vegasmansion

1 contributions

HU

HummerVan

1 contributions

IM

imokoi

1 contributions

SE

sencere

1 contributions

DI

dingmib

1 contributions

LI

lincappu

1 contributions

AL

alexnept13

1 contributions

LI

liqiang0330

1 contributions

MA

makefifo

1 contributions

TI

tiana2015

1 contributions

JA

Jabf-ht

1 contributions

HK

HkkSimple

1 contributions

GO

Goolv

1 contributions

ZJ

zjbsj01

1 contributions

IX

ixyxj

1 contributions

PZ

PzzAg6

1 contributions

SU

Suchana34

1 contributions

AR

ArkURL

1 contributions

JI

jiancn

1 contributions

LE

leavenotrace

1 contributions

SU

suainam

1 contributions

YU

yusufunlu

1 contributions

XB

xbronze

1 contributions

FS

Fsouth

1 contributions

TI

Tiresiasel

1 contributions

NI

NikolaZhang

1 contributions

DI

DiyGitAll

1 contributions

P4

P4nda0s

1 contributions

HU

HungryKonata

1 contributions

HX

HxXhY

1 contributions

LV

lvliuzhong

1 contributions

SA

satorielectronics

1 contributions

RU

runningrao

1 contributions

CH

ch0ux

1 contributions

LE

lenbolan

1 contributions

QF

Qffe

1 contributions

CY

cyd1310997

1 contributions

KI

Kimpten

1 contributions

HI

HinVec

1 contributions

CH

changzhongzhong

1 contributions

SM

smcclennon

1 contributions

HE

heeler-deer

1 contributions

BO

BOBO-LI3721

1 contributions

HY

Hy-1990

1 contributions

TR

TrojansChoice

1 contributions

NO

no-boundary

1 contributions

AU

austinlv

1 contributions

TA

tangminji

1 contributions

MI

Mithileshraj123

1 contributions

DJ

DJango996

1 contributions

HU

huang-a123

1 contributions

HU

huazidev

1 contributions

IH

iheb457gkk

1 contributions

JJ

JJOPED

1 contributions

12

128jku9j

1 contributions

MA

MartianDominic

1 contributions

XZ

xzAscC

1 contributions

NE

Networks-KH

1 contributions

SH

shinigam9

1 contributions

AM

ambitious-windx

1 contributions

YU

Yuanzidan081

1 contributions

MU

mumu-1005

1 contributions

RU

rucaiChow

1 contributions

LI

liuwwang

1 contributions

KL

KleytonRomao

1 contributions

LC

lccccch

1 contributions

Q2

q250305917

1 contributions

NE

NeoCh3n

1 contributions

K3

k3l7ON

1 contributions

MA

MadysonMccall

1 contributions

TM

tmquoc0104

1 contributions

XU

xuxia2013

1 contributions

FL

flueticjustice

1 contributions

TU

TuanTayHo

1 contributions

KA

kawayixixing

1 contributions

CI

cifangyiquan

1 contributions

NO

NocGeek

1 contributions

DA

danweiyuancircle

1 contributions

SY

SyYhunfhds

1 contributions

WO

Wonkspele

1 contributions

AN

andresdiazf

1 contributions

KU

kunulanjewar

1 contributions

KI

kianpaya

1 contributions

GI

Giankaur03

1 contributions

ZA

zainAli-rgb

1 contributions

AM

Aman16-S

1 contributions

NN

nn75

1 contributions

TO

tomsk-7

1 contributions

ZH

zhangjilei

1 contributions

Related Repositories

Archmonger/django-dbbackup

1,079 stars

QingdaoU/OnlineJudge

6,448 stars

IndigoResearch/textteaser

629 stars

Track developers from scrapy/dirbot

Join 1,000+ companies finding quality developer leads