No cap, the most aesthetic, high-velocity, and decoupled Dark Web intelligence engine on GitHub.
Engineered with asynchronous SOCKS5 swarms, multi-provider LLM threat contextualization, SQLite FTS5 search indexes, and modular regex extractors.
Capabilities • Features • Benchmarks • Meta-Search • Architecture • Quickstart • Usage • Disclaimer
| Vector | Feature | Description |
|---|---|---|
| 🧠 AI Intelligence | Multi-LLM Threat Triage | Integrated with Ollama (Llama 3), Google Gemini, OpenAI (GPT-4o), Anthropic (Claude 3.5) & OpenRouter. Provides deterministic offline heuristic fallbacks when offline. |
| 🧅 Dark Web Spider | Asynchronous Meta-Search | Concurrently scans 11+ Dark Web search engines (Ahmia, OnionLand, Torch, Kaizer, Amnesia, etc.) to discover hidden services prior to crawling. |
| ⚡ Extreme Velocity | Async SOCKS5 Swarm | Built on top of aiohttp + aiohttp-socks with rate-limiting, circuit renewal, non-blocking coroutines, and automated backoff. |
| 💰 Artifact Harvester | Crypto & PGP Extraction | Pluggable regex extractors harvest Bitcoin (bc1, legacy), Ethereum (0x), Monero (4/8), PGP public keys, emails, and phone numbers. |
| 🛠️ Tech Fingerprinter | Stack Identification | Detects 25+ web frameworks and servers: React, Next.js, Vue, Nuxt, FastAPI, Flask, Django, Express, WordPress, Nginx, TailwindCSS, etc. |
| 🔍 Search Engine | SQLite FTS5 Full-Text | Blazing-fast BM25 full-text search indexing across millions of scraped pages with sub-millisecond retrieval. |
| 📊 Cyber Reports | Dark Cyberpunk Exports | Automatically generates high-aesthetic HTML, JSON, and PDF forensic intelligence dossiers with copyable artifact badges. |
| 🌐 Modern Interfaces | FastAPI Dashboard + CLI | Switch effortlessly between a reactive web dashboard with live stats and an elite rich-powered interactive terminal console. |
How does DeepRecon stack up against traditional crawlers? DeepRecon utilizes asynchronous I/O and non-blocking SOCKS5 multiplexing to yield up to 10x higher node discovery throughput:
DeepRecon queries multiple curated onion search indexes concurrently to identify live services across disparate networks:
Each scanned onion node passes through a sub-millisecond decoupled intelligence pipeline:
DeepRecon/
├── core/ # Engine Core: SOCKS5 Async Crawler, AI Triage, Search Engines, Parser, Reporter
│ ├── ai_analyzer.py # Multi-LLM provider router (Ollama, Gemini, OpenAI, Anthropic, Heuristics)
│ ├── crawler.py # Async BFS crawler with rate-limiting & duplicate avoidance
│ ├── parser.py # HTML/LXML AST entity extractor & form parser
│ ├── reporter.py # Dark-mode Cyberpunk HTML, JSON, and PDF report builder
│ ├── search_engines.py # 11-Engine async Dark Web meta-search dispatcher
│ └── searcher.py # Regex & keyword intelligence scoring engine
├── plugins/ # Extensible plugin system (Drop any Python plugin here)
│ ├── crypto_detector.py # BTC (bc1/legacy), ETH, and Monero (XMR) sniffer
│ ├── email_extractor.py # Email regex pattern harvester
│ ├── fingerprinter.py # 25+ Web server, JS framework & CMS tech identifier
│ ├── language_detector.py # NLP natural language identification
│ └── pgp_harvester.py # PGP ASCII Armor public key block extractor
├── storage/ # Persistence Layer: SQLite FTS5 Full-Text Search
│ ├── db.py # Schema migration, BM25 text search, link graphs, relational tables
│ └── models.py # Dataclass entities (Site, Page, Link, KeywordHit, Session, Report)
├── utils/ # Utilities: Tor manager, OPSEC, Rate-limiter, Banner, Logger, Validator
├── web/ # Reactive FastAPI Web UI with Tailwind dark theme
└── tests/ # Comprehensive pytest test suite (100% passing)
██████╗ ███████╗███████╗██████╗ ██████╗ ███████╗ ██████╗ ██████╗ ███╗ ██╗
██╔══██╗██╔════╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝ ██╔═══██╗████╗ ██║
██║ ██║█████╗ █████╗ ██████╔╝██████╔╝█████╗ ██║ ██║ ██║██╔██╗ ██║
██║ ██║██╔══╝ ██╔══╝ ██╔═══╝ ██╔══██╗██╔══╝ ██║ ██║ ██║██║╚██╗██║
██████╔╝███████╗███████╗██║ ██║ ██║███████╗╚██████╗ ╚██████╔╝██║ ╚████║
╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═════╝ ╚═╝ ╚═══╝
⚡ Autonomous Dark Web OSINT & Intelligence Framework (v3.2.0)
╭─────────────────────── 🚀 Active Environment ───────────────────────╮
│ 🕵️ DeepRecon OSINT Intelligence Console │
│ 🧅 Tor Exit IP: 185.220.101.5 (Secure Tor Circuit) │
│ 💾 Database: storage/deeprecon.db │
│ 🤖 AI Provider: OLLAMA (Llama3) │
╰──────────────────────────────────────────────────────────────────────╯
1 Crawl direct target (.onion or clearnet)
2 Global Dark Web Meta-Search (11 Search Engines)
3 Search stored pages locally (FTS5 BM25 Engine)
4 Generate AI Threat Analysis on Session
5 Renew Tor IP (NEWNYM Signal)
6 Generate session report (HTML / JSON / PDF)
7 List sessions
8 Exit
Spin up the entire DeepRecon stack with isolated networking and persistence:
docker-compose up --build -dOpen
http://localhost:8000in your browser to access the Web UI.
# Clone the repository
git clone https://github.com/taezeem14/DeepRecon.git
cd DeepRecon
# Run the automated installer
chmod +x install.sh
./install.shgit clone https://github.com/taezeem14/DeepRecon.git
cd DeepRecon
install.batTip
Windows Tor Daemon: Ensure the Tor Expert Bundle is running on port 9050 (tor.exe).
Launch the FastAPI dark-mode dashboard for point-and-click intelligence operations:
deeprecon --web --port 8000Navigate to http://localhost:8000 to monitor live crawls, explore SQLite FTS5 search indexes, and trigger AI threat summaries.
Stay in the terminal with rich tables, color-coded outputs, and interactive prompts:
deeprecon --cliDeepRecon reads configurations from .env or system environment variables:
| Variable | Default | Description |
|---|---|---|
TOR_PROXY |
socks5h://127.0.0.1:9050 |
SOCKS5 proxy endpoint for Tor traffic routing |
TOR_CONTROL_PORT |
9051 |
Tor control port for NEWNYM circuit rotation |
CRAWL_WORKERS |
5 |
Concurrent async coroutines for crawling |
CRAWL_DEPTH |
2 |
Traversal depth limit for link graph discovery |
CRAWL_DELAY |
1.5 |
Throttling delay between requests per worker (sec) |
AI_PROVIDER |
ollama |
Provider: ollama, gemini, openai, anthropic, openrouter |
AI_MODEL |
llama3 |
Model identifier (e.g. llama3, gemini-1.5-flash, gpt-4o) |
GEMINI_API_KEY |
"" |
Google Gemini API Key |
OPENAI_API_KEY |
"" |
OpenAI API Key |
ANTHROPIC_API_KEY |
"" |
Anthropic API Key |
ENABLE_PDF_EXPORT |
false |
Enable automatic WeasyPrint PDF report rendering |
Run the automated pytest test suite (covers AI heuristics, SQLite FTS5, regex extractors, plugins, reporters, and FastAPI endpoints):
pytest -v======================== 18 passed in 8.52s ========================
Warning
This framework is developed strictly for authorized security research, defensive threat intelligence, and educational OSINT exploration. Interacting with arbitrary dark web services may carry legal and operational hazards depending on your jurisdiction. The authors and contributors assume no liability for misuse.
Mandatory OPSEC Rules:
- Always execute in dedicated virtualized sandbox environments.
- Verify Tor circuit status before commencing scraping.
- Never reuse personal credentials or clearnet identities.
Contributions make open source goated! Feel free to submit PRs for new search engines, plugin extractors, or frontend features:
- Fork the repo (
git checkout -b feature/EpicPlugin) - Commit your upgrades (
git commit -m 'feat: Add Monero subaddress tracker') - Push to your branch (
git push origin feature/EpicPlugin) - Open a Pull Request
If DeepRecon helped your research, smash that ⭐ — it keeps the project alive and motivates new features!
- Special credit to Apurv Singh Gautam for his foundational work on Robin, which inspired DeepRecon's AI integration and dark web search philosophy.
- The Tor Project for open-source anonymity networks.
- The FastAPI and Rich ecosystems for making terminal and web tooling effortless.
Distributed under the MIT License. See LICENSE for terms.