🛡️ Enterprise EDR System - Complete Documentation
https://img.shields.io/badge/License-MIT-yellow.svg https://img.shields.io/badge/python-3.9+-blue.svg https://img.shields.io/badge/docker-ready-blue.svg 🚀 Key Features 🔍 Advanced Detection
Real-time Monitoring: Process, network, file, registry, and memory monitoring
AI-Powered Analytics: Machine learning-based anomaly detection
Behavioral Analysis: Detection of suspicious process chains and activities
Threat Intelligence: Integration with multiple threat intelligence feeds
YARA Scanning: Memory and file scanning with custom rules
🛡️ Security Capabilities
Deception Technology: Honey tokens and canary objects
Persistence Detection: Registry, scheduled tasks, service monitoring
Lateral Movement Detection: SMB, RDP, WMI attack patterns
Memory Analysis: Real-time memory scanning for malware
Self-Protection: Agent integrity verification and anti-tampering
⚡ Automated Response
Playbook Engine: Automated incident response workflows
Endpoint Isolation: Automatic containment of compromised systems
Process Termination: Kill malicious processes automatically
Forensic Collection: Automated evidence gathering
🌐 Enterprise Ready
Scalable Architecture: Microservices-based design
Multi-tenant Support: Isolated environments for different teams
SIEM Integration: Splunk, ELK, ArcSight compatibility
SOAR Integration: Automated workflow orchestration
REST API: Comprehensive API for integration
🏗️ Architecture Overview text
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ EDR Agents │◄──►│ Load Balancer │◄──►│ API Servers │ │ (Endpoints) │ │ (HAProxy/Nginx)│ │ (FastAPI) │ └─────────────────┘ └──────────────────┘ └─────────────────┘ │ ▼ ┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ Web Dashboard │◄──►│ Frontend │◄──►│ Redis Cache │ │ (React) │ │ (Nginx) │ │ (Cluster) │ └─────────────────┘ └──────────────────┘ └─────────────────┘ │ ▼ ┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ SIEM Integration │◄──►│ Message Queue │◄──►│ PostgreSQL │ │ (Splunk/ELK) │ │ (RabbitMQ) │ │ (Cluster) │ └─────────────────┘ └──────────────────┘ └─────────────────┘
Technology Stack Backend Services
API Framework: FastAPI (Python 3.9+)
Message Queue: RabbitMQ 3.8+
Caching: Redis 6+
Database: PostgreSQL 14+
ORM: SQLAlchemy 2.0+
Authentication: JWT + OAuth2
Frontend
Framework: React 18 + TypeScript
State Management: Redux Toolkit
UI Library: Material-UI
Charts: Chart.js / D3.js
Real-time: WebSocket
Infrastructure
Containerization: Docker + Docker Compose
Orchestration: Kubernetes
Monitoring: Prometheus + Grafana
Logging: ELK Stack
CI/CD: GitHub Actions
🚀 Quick Start Prerequisites
Docker and Docker Compose
8GB RAM minimum, 16GB recommended
100GB free disk space
Development Deployment bash
git clone https://github.com/JULIASIV/EDS
then you must go to your directry which contains the cloned directry
docker-compose -f docker-compose.dev.yml up -d
docker-compose exec edr-server python -m server.db.init
Development URLs
Dashboard: http://localhost:3000
API Server: http://localhost:8000
API Docs: http://localhost:8000/api/docs
📋 System Requirements Server Requirements
OS: Ubuntu 20.04+, CentOS 8+, or Windows Server 2019+
CPU: 8 cores minimum, 16 cores recommended
RAM: 16GB minimum, 32GB recommended
Storage: 500GB SSD with 10k+ IOPS
Network: 1Gbps minimum
Agent Requirements
OS: Windows 10/11, Windows Server 2016+
CPU: 2 cores minimum
RAM: 2GB minimum
Storage: 1GB free space
Supported Integrations
SIEM: Splunk, Elastic Stack, IBM QRadar
SOAR: Demisto, Phantom, Swimlane
Threat Intelligence: VirusTotal, AlienVault OTX, AbuseIPDB
Notification: Slack, Microsoft Teams, PagerDuty
🛠️ Configuration Basic Configuration yaml
server: host: 0.0.0.0 port: 8000 workers: 4
database: url: postgresql://user:pass@localhost/edr pool_size: 20
redis: url: redis://localhost:6379
security: secret_key: your-secret-key agent_registration_token: your-registration-token
Detection Rules yaml
- name: "Process Hollowing Detection"
description: "Detects process hollowing techniques"
severity: HIGH
conditions:
- field: parent_process operator: in value: ["explorer.exe", "svchost.exe"]
- field: process_name operator: in value: ["notepad.exe", "calc.exe"]
📊 Monitoring & Alerting Alert Severity Levels
CRITICAL: Immediate response required
HIGH: Investigate within 1 hour
MEDIUM: Investigate within 4 hours
LOW: Review during normal operations
Notification Channels
Email (SMTP)
Slack Webhooks
Microsoft Teams
PagerDuty
Custom Webhooks
🔧 Usage Examples Register an Agent bash
curl -X POST https://edr.company.com/api/v1/agents/register
-H "Content-Type: application/json"
-d '{
"hostname": "workstation-01",
"ip_address": "192.168.1.100",
"os_version": "Windows 10",
"registration_token": "your-token"
}'
Search for Alerts python
from edr.client import EDRClient
client = EDRClient(api_key="your-api-key") alerts = client.search_alerts( query="severity:HIGH AND event_type:malware_detection", timeframe="24h" )
Create Hunting Rule python
rule = { "name": "Suspicious PowerShell Activity", "query": 'process_name:"powershell.exe" AND command_line:"-enc"', "description": "Detects encoded PowerShell commands" } client.create_hunting_rule(rule)
🚀 Deployment Guide Production Deployment Option 1: Docker Swarm bash
docker swarm init
docker stack deploy -c docker-compose.prod.yml edr
docker service scale edr_edr-server=3 docker service scale edr_edr-worker=5
Option 2: Kubernetes bash
helm install edr edr/edr-system
--namespace edr-system
--values values-production.yaml
Agent Deployment Windows Deployment powershell
Start-Process msiexec -ArgumentList @( "/i", "EDR-Agent.msi", "/quiet", "/norestart", "SERVER_URL=https://edr.company.com", "REGISTRATION_TOKEN=your-token" ) -Wait
Linux Deployment bash
wget https://edr.company.com/downloads/edr-agent_2.0.0_amd64.deb dpkg -i edr-agent_2.0.0_amd64.deb
systemctl enable edr-agent systemctl start edr-agent
🏗️ System Architecture Component Design EDR Agent Architecture text
EDR Agent (Endpoint) ├── Core Engine │ ├── Configuration Manager │ ├── Communication Manager │ ├── Event Collector │ └── Response Executor ├── Monitoring Modules │ ├── Process Monitor (ETW/WMI) │ ├── Network Monitor (Windows API) │ ├── File System Monitor (Watchdog) │ ├── Registry Monitor (WinReg) │ ├── Memory Scanner (YARA) │ └── Persistence Detector ├── Security Features │ ├── Self-Protection │ ├── Integrity Verification │ ├── Anti-Tampering │ └── Secure Communication └── Data Management ├── Event Queue ├── Local Cache ├── Log Manager └── Update Handler
Detection Engine Architecture text
Detection Engine ├── Layer 1: Signature-Based │ ├── YARA Rule Scanner │ ├── IOC Matcher │ └── Hash Reputation ├── Layer 2: Behavior-Based │ ├── Process Lineage Analysis │ ├── Network Behavior Profiling │ ├── File Activity Patterns │ └── Registry Modification Tracking ├── Layer 3: AI-Powered │ ├── Anomaly Detection (Isolation Forest) │ ├── Behavioral Clustering (DBSCAN) │ ├── Command Line Analysis (TF-IDF) │ └── Temporal Pattern Recognition └── Layer 4: Deception-Based ├── Honey Token Monitoring ├── Canary File Access Detection ├── Decoy Service Monitoring └── Trap Credential Usage
Data Flow Event Collection Flow text
Endpoint Event → Agent Collection → Local Processing → Secure Transmission → Message Queue → Event Processing → Detection Engine → Alert Generation → Storage → Dashboard
Alert Processing Flow text
Detection Trigger → Alert Creation → Enrichment → Correlation → Severity Assessment → Notification → Response Actions → Incident Creation
🔌 API Reference Authentication Get Authentication Token http
POST /auth/token Content-Type: application/x-www-form-urlencoded
username=admin&password=your_password
Response: json
{ "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9...", "token_type": "bearer", "expires_in": 3600 }
Agents API Register Agent http
POST /agents/register Content-Type: application/json X-EDR-Registration-Token: your-registration-token
{ "hostname": "workstation-01", "ip_address": "192.168.1.100", "os_version": "Windows 10.0.19044", "agent_version": "2.0.0" }
Submit Heartbeat http
POST /agents/{agent_id}/heartbeat Authorization: Bearer {agent_token}
{ "timestamp": "2024-01-15T10:30:00Z", "system_info": { "cpu_usage": 45.2, "memory_usage": 1234567890, "disk_usage": 85.5 } }
Events API Submit Events http
POST /events Authorization: Bearer {agent_token} Content-Type: application/json
[ { "event_type": "process_creation", "timestamp": "2024-01-15T10:30:00Z", "agent_id": "agent-123456", "data": { "process_name": "notepad.exe", "process_id": 1234, "parent_process_id": 567, "command_line": "notepad C:\temp\file.txt" } } ]
Search Events http
GET /events?event_type=process_creation&severity=HIGH&from=2024-01-15T00:00:00Z Authorization: Bearer {admin_token}
Alerts API List Alerts http
GET /alerts?severity=HIGH&status=open&from=2024-01-15T00:00:00Z Authorization: Bearer {admin_token}
Execute Response Action http
POST /alerts/{alert_id}/actions Authorization: Bearer {admin_token} Content-Type: application/json
{ "action_type": "isolate_endpoint", "parameters": { "duration_minutes": 60 } }
🔍 Detection Rules & Capabilities Rule Format Basic Rule Structure yaml
- name: "Rule Name"
description: "Rule description"
severity: "HIGH"
enabled: true
category: "process"
conditions:
- field: "process_name" operator: "equals" value: "malware.exe"
- field: "parent_process" operator: "in" value: ["explorer.exe", "svchost.exe"] actions:
- type: "alert" parameters: message: "Malicious process detected"
- type: "isolate_endpoint" parameters: duration: 3600
Process Detection Process Injection Detection yaml
- name: "Process Hollowing Detection"
description: "Detects process hollowing techniques"
severity: "HIGH"
enabled: true
conditions:
- field: "parent_process" operator: "in" value: ["explorer.exe", "svchost.exe", "services.exe"]
- field: "process_name" operator: "in" value: ["notepad.exe", "calc.exe", "winword.exe"]
- field: "integrity_level" operator: "equals" value: "Medium"
PowerShell Obfuscation Detection yaml
- name: "Obfuscated PowerShell Detection"
description: "Detects encoded or obfuscated PowerShell commands"
severity: "HIGH"
enabled: true
conditions:
- field: "process_name" operator: "equals" value: "powershell.exe"
- field: "command_line" operator: "regex" value: ".-enc\s+[A-Za-z0-9+/=]+."
Network Detection C2 Communication Patterns yaml
- name: "Beaconing Behavior Detection"
description: "Detects regular outbound connections indicating C2"
severity: "HIGH"
enabled: true
conditions:
- field: "connection_count" operator: "greater_than" value: 50
- field: "connection_interval_stddev" operator: "less_than" value: 5
Data Exfiltration Detection yaml
- name: "Large Outbound Transfer"
description: "Detects large data transfers to external IPs"
severity: "HIGH"
enabled: true
conditions:
- field: "bytes_sent" operator: "greater_than" value: 104857600 # 100MB
- field: "remote_ip" operator: "regex" value: "^(?!192\.168|10\.|172\.(1[6-9]|2[0-9]|3[0-1]))"
File System Detection Ransomware Patterns yaml
- name: "Ransomware File Activity"
description: "Detects ransomware-like file modification patterns"
severity: "CRITICAL"
enabled: true
conditions:
- field: "file_operations" operator: "greater_than" value: 1000
- field: "file_extension_changes" operator: "greater_than" value: 50
- field: "time_window" operator: "less_than" value: 300 # 5 minutes
🔗 Integration Guide SIEM Integration Splunk Integration python
class SplunkIntegration: def init(self, config): self.hec_url = f"{config['base_url']}/services/collector/event" self.hec_token = config['hec_token'] self.headers = { 'Authorization': f'Splunk {self.hec_token}', 'Content-Type': 'application/json' }
async def send_alert(self, alert):
event = {
'event': alert,
'sourcetype': 'edr:alert',
'source': 'edr-system',
'host': alert.get('agent_id', 'unknown')
}
response = requests.post(
self.hec_url,
headers=self.headers,
json=event,
verify=False
)
Elastic Stack Integration python
class ElasticIntegration: def init(self, config): self.es = Elasticsearch( [config['host']], http_auth=(config['username'], config['password']), scheme="https", port=9200 )
async def send_events(self, events):
actions = []
for event in events:
action = {
"_index": f"edr-{event['event_type']}",
"_source": event
}
actions.append(action)
success, failed = bulk(self.es, actions)
Threat Intelligence VirusTotal Integration python
class VirusTotalIntegration: def init(self, config): self.api_key = config['api_key'] self.base_url = 'https://www.virustotal.com/api/v3' self.headers = {'x-apikey': self.api_key}
async def check_file_hash(self, file_hash):
async with aiohttp.ClientSession() as session:
async with session.get(
f"{self.base_url}/files/{file_hash}",
headers=self.headers
) as response:
if response.status == 200:
data = await response.json()
return self.parse_response(data)
Notification Services Slack Integration python
class SlackIntegration: def init(self, config): self.webhook_url = config['webhook_url'] self.channel = config.get('channel', '#security-alerts')
async def send_alert(self, alert):
message = self.format_message(alert)
payload = {
'channel': self.channel,
'username': 'EDR System',
'text': message,
'icon_emoji': ':warning:'
}
async with aiohttp.ClientSession() as session:
await session.post(self.webhook_url, json=payload)
🔧 Troubleshooting Guide Quick Diagnosis Health Check Script bash
#!/bin/bash echo "=== EDR System Health Check ==="
curl -f https://edr.company.com/health > /dev/null 2>&1 if [ $? -eq 0 ]; then echo "✓ Server is reachable" else echo "✗ Server is not reachable" fi
docker-compose exec postgres pg_isready -U edr_user if [ $? -eq 0 ]; then echo "✓ Database is healthy" else echo "✗ Database connection failed" fi
docker-compose exec redis redis-cli ping | grep -q PONG if [ $? -eq 0 ]; then echo "✓ Redis is healthy" else echo "✗ Redis connection failed" fi
Common Issues Agent Not Registering
Symptoms:
Agent fails to connect to server
No heartbeat received
Agent not visible in dashboard
Solutions:
Verify server URL in configuration
Check registration token
Ensure outbound 443/TCP is allowed
Configure proxy if required
High Resource Usage
Symptoms:
High CPU or memory usage by agent process
System performance degradation
Solutions:
Adjust scan intervals:
yaml
monitoring: process_scan_interval: 5 # Increase from 2 to 5 seconds network_scan_interval: 10 # Increase from 5 to 10 seconds
Disable intensive modules:
yaml
modules: memory_scanner: false deep_file_analysis: false
Performance Issues High Server Load
Symptoms:
Slow API responses
High CPU/memory usage
Queue backlogs
Solutions:
Scale horizontally:
yaml
services: edr-server: deploy: replicas: 3 # Increase from 1 to 3 edr-worker: deploy: replicas: 5 # Increase from 2 to 5
Optimize database:
sql
CREATE INDEX CONCURRENTLY idx_events_timestamp ON events (timestamp DESC);
CREATE INDEX CONCURRENTLY idx_alerts_severity_status ON alerts (severity, status);
🗺️ Development Roadmap Phase 1: Enhanced Detection (0-3 Months)
Network XDR Module
Firewall log integration
Network traffic analysis
Lateral movement detection
Cloud Security Module
AWS CloudTrail integration
Azure Security Center feeds
Email Security Integration
Office 365 threat intelligence
Phishing campaign correlation
Phase 2: Advanced AI (3-6 Months)
Deep Learning Engine
Behavioral anomaly detection
Sequence analysis models
Multi-modal correlation
NLP Security Analysis
Command line intent analysis
Threat intelligence correlation
Autonomous Response
AI-generated playbooks
Predictive containment
Phase 3: Enterprise Features (6-12 Months)
Multi-tenancy & RBAC
Tenant isolation
Advanced role-based access control
Advanced Analytics
Threat hunting workbench
Custom dashboard builder
Compliance & Reporting
Regulatory compliance templates
Automated compliance reporting
Phase 4: Advanced Capabilities (12-18 Months)
Zero Trust Integration
Device health attestation
Continuous authentication
Dynamic access policies
Advanced Deception
AI-generated honeypots
Dynamic deception environments
Attack intelligence gathering
MDR Services Platform
Managed detection and response
Security operations center integration
Threat intelligence sharing
🤝 Contributing
We welcome contributions! Please see our Contributing Guide for details. Development Setup bash
git clone https://github.com/your-username/edr-system.git
python -m venv venv source venv/bin/activate pip install -r requirements-dev.txt
pytest tests/ -v
black server/ agent/ mypy server/ agent/
📄 License
This project is licensed under the MIT License - see the LICENSE file for details. 🆘 Support
Documentation: Full Documentation
Community Forum: GitHub Discussions
Issue Tracker: GitHub Issues
Security Issues: security@your-org.com
🙏 Acknowledgments
Thanks to the open-source security community
Contributors and beta testers
Security researchers who provided feedback
Enterprise EDR System - Protecting your endpoints with advanced detection and response capabilities.