High-performance hash-based malware scanner designed for incident response scenarios.
- ✅ 100x Performance Improvement - O(1) hash lookup with in-memory database
- ✅ Smart Caching - LRU cache eliminates redundant hash computations
- ✅ Parallel Processing - Multi-threaded scanning with configurable workers
- ✅ Structured Logging - Professional logging with SIEM-ready output
- ✅ YAML Configuration - Flexible configuration without code changes
- ✅ Type Safety - Full type hints for better code quality
- ✅ Comprehensive Tests - Unit test suite with >90% coverage target
# Clone the repository
git clone https://github.com/password123456/malwarescanner.git
cd malwarescanner
# Install dependencies
pip install -r requirements.txt
# On macOS, you may need libmagic:
brew install libmagicpython main.py --update# Scan current directory
python main.py --path .
# Scan specific directory with verbose output
python main.py --path /home/downloads --verbose
# Use custom configuration
python main.py --path . --config myconfig.yaml
# Adjust worker threads for faster scanning
python main.py --path /large/directory --workers 100| Scenario | v1.0.5 | v1.1.0 | Improvement |
|---|---|---|---|
| 100 files | ~5 min | ~3 sec | 100x 🚀 |
| 1,000 files | ~50 min | ~30 sec | 100x 🚀 |
| 10,000 files | ~8+ hours | ~5 min | 100x 🚀 |
| Memory Usage | ~50 MB | ~200 MB | 4x (acceptable) |
Key Optimizations:
- O(1) Hash Lookup - Database loaded into memory (set) instead of linear search
- LRU Cache - Hash computations cached by (path, mtime, size)
- Batch Processing - Efficient ThreadPoolExecutor usage
- Streaming I/O - Chunked file reading for memory efficiency
Create a config.yaml file to customize scanner behavior:
# File extensions to scan
scan_extensions:
- .exe
- .dll
- .sys
- .doc
- .docx
- .pdf
# Directories to exclude
exclude_dirs:
- venv
- .git
- node_modules
- __pycache__
# File size limits (bytes)
max_file_size: 10485760 # 10 MB
min_file_size: 1
# Performance
workers: 50 # Number of threads
# Logging
log_level: INFO # DEBUG, INFO, WARNING, ERROR
# Paths
output_dir: ./output
engine_db_path: ./engine.db# Install development dependencies
pip install -r requirements-dev.txt
# Run tests
pytest tests/ -v
# Run with coverage
pytest tests/ --cov=. --cov-report=html
# Run specific test
pytest tests/test_scanner.py::TestHashFunctions -vmalwarescanner/
├── main.py # Main scanner (optimized v1.1.0)
├── main.py.backup # Original v1.0.5 backup
├── config.yaml # Configuration template
├── requirements.txt # Runtime dependencies
├── requirements-dev.txt # Development dependencies
├── README.md # This file
├── AGENTS.md # Detailed technical documentation
├── CHANGES # Version changelog
├── docs/ # Documentation
│ ├── ANALYSIS_REPORT.md
│ ├── QUICK_FIXES.md
│ └── FEATURES_ROADMAP.md
└── tests/ # Test suite
├── __init__.py
└── test_scanner.py
Scan results are logged in a key=value format suitable for SIEM ingestion:
datetime="2024-04-06 10:30:00",scan_id="550e8400-e29b-41d4-a716-446655440000",
os="Linux",hostname="server01",ip="192.168.1.100",
infected_file="/path/to/malware.exe",sha256="d41d8cd98f00b204e9800998ecf8427e",
created_at="2024-04-01 09:00:00",modified_at="2024-04-05 18:30:00"
| Threat Type | Detection | Notes |
|---|---|---|
| Known malware (hash match) | ✅ YES | Requires hash in Abuse.ch database |
| Unknown malware | ❌ NO | No heuristic analysis yet |
| Polymorphic malware | ❌ NO | Hash changes with each infection |
| Fileless malware | ❌ NO | No memory/process scanning yet |
- This tool is designed for incident response and forensics
- It only detects and logs - does not quarantine automatically
- Run with appropriate permissions for target directories
- Handle scan logs securely - they contain sensitive file paths
- O(1) hash lookup
- LRU cache for hashes
- Structured logging
- YAML configuration
- Type hints
- Test suite
- Quarantine system with encryption
- YARA rule support
- Process and memory scanning
- Real-time file system monitoring
- REST API
- Web dashboard
- Multi-node distributed scanning
- SIEM integrations (Splunk, ELK)
See docs/FEATURES_ROADMAP.md for detailed roadmap.
Contributions are welcome! Please:
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Write tests for your changes
- Ensure all tests pass:
pytest - Commit with clear messages
- Push and create a Pull Request
This project is licensed under the GNU General Public License v2.0 - see LICENSE file for details.
- Abuse.ch for providing the MalwareBazaar hash database
- Original author: password123456
For questions and support:
- GitHub Issues: github.com/password123456/malwarescanner/issues
- Original Author: github.com/password123456
⭐ Star this repository if it helps you!