feat: integrate uncover multi-engine search, CDN/IP filtering, and consolidate Censys credentials#82
Merged
samugit83 merged 1 commit intosamugit83:masterfrom Mar 31, 2026
Conversation
…nsolidate Censys credentials - Add ProjectDiscovery uncover integration across pipeline, agent, and UI with configurable max results and per-engine API key management - Add CDN/Cloudflare/CloudFront IP filtering (ip_filter.py) to all OSINT enrichment modules (Shodan, Censys, FOFA, Netlas, OTX, VirusTotal, ZoomEye) - Harden Criminal IP enrichment with proper error classification, credit exhaustion detection, and single-warning-then-stop behavior - Consolidate 4 Censys credential fields into 2 (API Token + Org ID) and migrate censys_enrich.py from deprecated Search API v2 to Platform API v3 - Add 10 new API key fields in Global Settings for uncover engines (Quake, Hunter, PublicWWW, HunterHow, Google, Onyphe, Driftnet, Censys Platform) - Wire uncover as GROUP 2b target expansion phase in recon pipeline (before port scan) with Neo4j graph storage via update_graph_from_uncover mixin - Add UncoverToolManager to agentic layer with Docker-based execution - Add comprehensive unit tests (test_uncover_enrich.py) and update existing test suites for new key names and filtering behavior Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ip_filter.py) applied to all OSINT enrichment modules, preventing non-routable and CDN IPs from consuming API creditscensys_enrich.pyfrom deprecated Search API v2 to Platform API v3Type of Change
Component(s)
How to Test
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d postgres neo4j recon-orchestrator kali-sandbox agent webappdocker compose exec webapp npx prisma db pushpython3 -m pytest tests/test_censys_enrich.py tests/test_uncover_enrich.py tests/test_criminalip_enrich.py tests/test_graph_db_refactor.py -vChecklist
docker composeKey Changes by File
New Files
recon/ip_filter.pyrecon/uncover_enrich.pytests/test_uncover_enrich.pywebapp/prisma/migrations/20260329120000_consolidate_censys_keys/censys_api_idandcensys_api_secretcolumnsModified Files (31 total)
project_settings.py(key wiring),main.py(GROUP 2b uncover phase), all*_enrich.pymodules (CDN filtering),censys_enrich.py(v2→v3 migration)tools.py(UncoverToolManager + CensysToolManager v3),orchestrator.py(key wiring),prompts/tool_registry.pyosint_mixin.py(update_graph_from_uncovermethod)entrypoint.sh+container_manager.py(uncover image pull)