Crawling y Fuzzing web

---

1. Fuzzing de directorios y archivos

• ffuf

• feroxbuster

• gobuster

• wfuzz

---

2. Crawling web

2.1 Ejecución de crawling

• gau --threads 2 https://subdomain.domain.tld --o gau.txt 2>/dev/null

• katana -u https://subdomain.domain.tld -o katana1.txt

• katana -u https://subdomain.domain.tld -jc -js-crawl -kf robotstxt,sitemapxml,securitytxt -d 6 -o katana2.txt

• echo "https://subdomain.domain.tld" | hakrawler -d 6 -t 2 -u -i | tee hakrawler.txt

• crawlerx -d subdomain.domain.tld -o crawlerx.tx

• echo "subdomain.domain.tld" | waybackurls > waybackurls.txt

• cat crawlerx.txt gau.txt hakrawler.txt katana1.txt katana2.txt waybackurls.txt | sort -u > precrawling.txt

• rm crawlerx.txt gau.txt hakrawler.txt katana1.txt katana2.txt waybackurls.txt

• cat precrawling.txt | sed -n '/^https\?:\/\/subdomain\.domain\.tld/p' > crawling.txt

2.2 Filtros de crawling

a. Por strings

| grep -iE '\?|=|\?\*=|\?page(=)?|\?url(=)?|=\*\?|access|action|admin(istration)?|api|auth(entication)?|callbacks|cmd|debug|file|href|id(entification)?|image|img|jwt|location|login|next|page|path|q|query|\?query=|redirect(_url)|referer|return|search|secret|token|uri|url|v1|v2|v3|wp(-admin)?'

b. Por extensiones de archivos

| grep -iE '\.asp(x)?|\.bak|\.bash|\.sh|\.bat|\.cfm|\.cgi|\.conf|\.css|\.csv|\.db|\.do|\.ejs|\.env|\.exe|\.gif|\.gz|\.htaccess|\.htm(l)?|\.ico|\.ini|\.jar|\.jpg|\.jpeg|\.js(on)?|\.jsp(x)?|\.log|\.old|\.php(5)|\.pl|\.png|\.ps1|\.py|\.rb|\.sql(ite)?|\.svc|\.svg|\.swp|\.tar(.gz)?|\.ts|\.txt|\.woff(2)?|\.xml|\.zip'

---