Linux - ELF ViT Radar

Lightweight Vision Transformer for Linux ELF Malware Detection on Edge Devices

A unified end-to-end pipeline that detects and classifies Linux ELF malware using binary visualization and lightweight Vision Transformers (ViT). The system converts raw ELF binaries into multi-channel image representations, trains a DeiT-Tiny–class ViT alongside CNN baselines, and evaluates adversarial robustness — all in a single reproducible pipeline.

---

Table of Contents

• Motivation
• Research Foundation
• Architecture Overview
• Pipeline Steps
  • Step 1 — ELF Feature Extraction
  • Step 2 — Binary Visualization
  • Step 3 — ViT + CNN Training
  • Step 4 — Evaluation Dashboard
  • Step 5 — Adversarial Robustness
• Project Structure
• Installation
• Usage
• Vendor Repositories
• Model Details
• Configuration Reference
• Datasets
• References
• License

---

Motivation

The proliferation of Linux-based IoT devices has made ELF (Executable and Linkable Format) binaries a primary attack vector for botnets like Mirai, Gafgyt, and Tsunami. Traditional signature-based antivirus engines are ineffective on resource-constrained edge devices, and the volume of new malware variants demands automated, learning-based classification.

This project merges two complementary research directions:

1. Static analysis of ELF binaries — extracting structural features (sections, segments, imports, entropy, byte histograms) from Linux malware without executing the binary.
2. Vision Transformer–based classification — converting binary data into images and leveraging the self-attention mechanism of ViTs to capture global structural patterns that CNNs miss.

The result is a lightweight, edge-deployable malware classifier that operates entirely through static analysis — no sandbox, no dynamic instrumentation, no network connectivity required.

---

Research Foundation

This project is directly inspired by and builds upon the following published work:

Paper	Venue	Year	Relevance
ViT4Mal: Lightweight Vision Transformer for Malware Detection on Edge Devices — Akshara Ravi, Vivek Chaturvedi, Muhammad Shafique	ACM Transactions on Embedded Computing Systems (TECS) / CASES ESWEEK	2023	Core architecture — ViT for malware image classification on edge
Malware Analysis using ELF features for Linux-based IoT devices — Akshara Ravi, Vivek Chaturvedi	IEEE VLSID	2022	Feature extraction methodology — 30 ELF features, family labeling
FRNet: A Feature-Rich CNN Architecture to Defend against Adversarial Attacks	IEEE Access	2024	Adversarial defense strategy, feature-rich CNN design principles
S-E Pipeline: ViT-Based Resilient Classification for Medical Imaging Against Adversarial Attacks	IEEE IJCNN	2024	ViT adversarial robustness, self-ensembling defense concept
Fast and Efficient Decision-based Attack for DNN on Edge — H. Jain et al.	IEEE SIPS	2020	Edge-device attack model

---

Architecture Overview

┌──────────────────────────────────────────────────────────────────────┐
│                        ELF-ViT-Mal Pipeline                         │
│                                                                      │
│  ┌─────────────┐   ┌──────────────┐   ┌───────────────┐             │
│  │  Raw ELF     │   │  Feature     │   │   Image       │             │
│  │  Binaries    │──▶│  Extraction  │──▶│   Generation  │             │
│  │  (data/raw/) │   │  (pyelftools)│   │   (3-channel) │             │
│  └─────────────┘   └──────────────┘   └───────┬───────┘             │
│                                                │                     │
│                         ┌──────────────────────┼──────────────┐      │
│                         ▼                      ▼              ▼      │
│                   ┌──────────┐          ┌──────────┐   ┌──────────┐  │
│                   │  ViT     │          │ ResNet18 │   │MobileNet │  │
│                   │ (DeiT-   │          │ Baseline │   │V2 Basel- │  │
│                   │  Tiny)   │          │          │   │ine       │  │
│                   └────┬─────┘          └────┬─────┘   └────┬─────┘  │
│                        │                     │              │        │
│                        └─────────┬───────────┘──────────────┘        │
│                                  ▼                                   │
│                         ┌────────────────┐                           │
│                         │   Evaluation   │                           │
│                         │   Dashboard    │──▶  Confusion Matrices    │
│                         │                │──▶  ROC Curves            │
│                         │                │──▶  Attention Maps        │
│                         └───────┬────────┘                           │
│                                 ▼                                    │
│                         ┌────────────────┐                           │
│                         │  Adversarial   │──▶  FGSM / PGD / BIM     │
│                         │  Robustness    │──▶  Clean vs Attacked     │
│                         └────────────────┘     Accuracy Report       │
└──────────────────────────────────────────────────────────────────────┘

---

Pipeline Steps

Step 1 — ELF Feature Extraction

Module: src/pipeline/step1_elf_feature_extraction.py

Processes raw ELF binaries using pyelftools (pure Python — no shell commands, no MySQL dependency). Extracts 30+ features per binary:

• Header features: ELF class (32/64-bit), endianness, ABI, machine type, entry point address
• Section analysis: .text size, .data size, .rodata size, .bss size, section count, section entropy, writable + executable section flags (W^X check)
• Segment analysis: segment count, presence of PT_DYNAMIC / PT_INTERP / PT_GNU_STACK
• Import/export tables: total import count, security-sensitive function counts (network, process, file, crypto, anti-debug categories)
• String analysis: total strings, suspicious string count (IP patterns, URLs, shell commands), average string length
• Binary-level features: Shannon entropy, byte histogram (256-bin), printable character ratio

Output: data/processed/elf_features.csv + per-binary byte histograms.

Step 2 — Binary Visualization

Module: src/pipeline/step2_binary_visualization.py

Converts ELF binaries into 224×224 images using three complementary visualization methods, then combines them into a single composite input:

Channel	Method	Description
R (Red)	byte_plot	Direct byte-to-pixel mapping — each byte value (0–255) becomes a pixel intensity. Binary is resized/tiled to fill the image.
G (Green)	entropy_plot	Section-wise entropy heatmap — computes local Shannon entropy across sliding windows, visualizing information density.
B (Blue)	feature_matrix	Feature co-occurrence matrix — inspired by MalwareViT's opcode co-occurrence approach, but computed from byte bigrams.

The composite RGB image fuses all three views into a single 3-channel input, giving the ViT richer structural information than any single visualization alone.

Images are organized as ImageFolder structure for PyTorch:

data/images/{composite,grayscale}/{train,test}/{family}/*.png

Step 3 — ViT + CNN Training

Module: src/pipeline/step3_vit_classifier.py

Trains multiple models in parallel on a shared dataset:

Model	Parameters	Description
MalwareViT (DeiT-Tiny)	~5.7M	Custom ViT: 192 embed dim, 12 layers, 3 heads, 16×16 patches. Returns attention weights for visualization.
MalwareViT (Micro)	~1.2M	Ultra-lightweight: 128 embed dim, 6 layers, 4 heads. Designed for edge deployment.
ResNet18	~11.2M	Pretrained on ImageNet, fine-tuned. First conv adapted for composite input.
MobileNetV2	~2.2M	Pretrained, lightweight. Edge-deployment baseline.
SimpleCNN	~0.5M	3-conv-layer network matching vendor repo architecture. Minimal baseline.

Training features:

• Optimizer: AdamW with weight decay
• Scheduler: Cosine annealing with linear warmup (5 epochs)
• Early stopping: Patience 10, monitoring validation loss
• Gradient clipping: Max norm 1.0
• Data augmentation: Random horizontal flip, rotation (±10°), color jitter
• Checkpointing: Saves best model per validation loss

Step 4 — Evaluation Dashboard

Module: src/pipeline/step4_evaluation.py

Generates comprehensive evaluation artifacts:

• Confusion matrices (per model) — heatmap with actual vs predicted family labels
• ROC curves — per-class and macro-average AUC for multi-class classification
• Training history plots — loss and accuracy curves across epochs
• Attention map visualization — extracts and overlays ViT self-attention weights on input images, showing which binary regions the model focuses on
• Model comparison table — side-by-side accuracy, F1, parameters, inference time

All outputs saved to results/figures/, results/reports/, results/attention_maps/.

Step 5 — Adversarial Robustness (Bonus)

Module: src/pipeline/step5_adversarial.py

Tests trained models against adversarial perturbations using torchattacks:

Attack	Type	Parameters
FGSM	Single-step	ε = 0.03
PGD	Iterative	ε = 0.03, α = 0.007, steps = 10
BIM	Iterative	ε = 0.03, α = 0.007, steps = 10
CW	Optimization-based	c = 1.0, κ = 0, steps = 100, lr = 0.01

Outputs:

• Clean accuracy vs. adversarial accuracy per model per attack
• Adversarial example visualizations (clean → perturbation → adversarial)
• JSON robustness report at results/reports/adversarial_robustness.json

This step bridges two active research clusters — malware detection (Cluster 1) and adversarial ML (Cluster 2) — demonstrating that malware classifiers are themselves vulnerable to evasion attacks.

---

Project Structure

cybersecurity/
├── README.md                          # This file
├── requirements.txt                   # Python dependencies
├── pyproject.toml                     # Project metadata & build config
├── .gitignore                         # Git exclusions
│
├── IIT PKD/                           # Research context documents
│   ├── dr-vivek.md                    #   Faculty profile
│   ├── findings.md                    #   Research analysis & repo intelligence
│   └── prompt.md                      #   Application context
│
├── src/                               # Core pipeline source code
│   ├── __init__.py
│   ├── config.py                      #   Centralized configuration (all hyperparams)
│   │
│   ├── pipeline/                      #   Pipeline step implementations
│   │   ├── step1_elf_feature_extraction.py
│   │   ├── step2_binary_visualization.py
│   │   ├── step3_vit_classifier.py
│   │   ├── step4_evaluation.py
│   │   └── step5_adversarial.py
│   │
│   ├── models/                        #   Model architectures
│   │   ├── vit.py                     #     Vision Transformer (DeiT-Tiny / Micro)
│   │   └── cnn_baseline.py            #     ResNet18, MobileNetV2, SimpleCNN
│   │
│   ├── utils/                         #   Shared utilities
│   │   ├── elf_parser.py              #     Pure-Python ELF feature extraction
│   │   ├── image_utils.py             #     Binary-to-image conversion
│   │   └── metrics.py                 #     Metrics, plotting, attention maps
│   │
│   └── data/                          #   Dataset utilities
│       └── dataset.py                 #     PyTorch DataLoader creation
│
├── scripts/                           #   CLI tools
│   ├── run_pipeline.py                #   Main pipeline runner (argparse CLI)
│   └── download_samples.py            #   Dataset download helper
│
├── ui/                                #   Web dashboard (Streamlit)
│   └── app.py                         #     Main web UI entry point
│
├── vendors/                           #   Cloned reference repositories
│   ├── elf-static-analysis/           #     ashalaginov/Linux-ELF-malware-static-analysis
│   ├── malware-image-detection/       #     TanayBhadula/malware-image-detection
│   └── MalwareViT/                    #     rickyxume/MalwareViT
│
├── data/                              #   Runtime data (git-ignored)
│   ├── raw/{family}/                  #     Raw ELF binaries per family
│   ├── processed/                     #     Extracted features (CSV)
│   ├── images/{method}/{split}/{family}/  # Generated visualization images
│   └── models/                        #     Trained model checkpoints (.pth)
│
├── results/                           #   Pipeline outputs (git-ignored)
│   ├── figures/                       #     Confusion matrices, ROC curves, plots
│   ├── reports/                       #     JSON/text evaluation reports
│   └── attention_maps/                #     ViT attention visualizations
│
├── notebooks/                         #   Jupyter notebooks (exploratory)
└── tests/                             #   Unit tests

---

Installation

Prerequisites

• Python 3.9+
• pip or conda
• (Optional) CUDA-enabled GPU for faster training
• (Optional) Apple Silicon Mac with MPS support

Setup

# Clone the repository
git clone https://github.com/mantavyam/ELF-ViT-Mal.git
cd ELF-ViT-Mal

# Create virtual environment
python -m venv .venv
source .venv/bin/activate   # macOS/Linux

# Install dependencies
pip install -r requirements.txt

Verify installation

python -c "import torch; print(f'PyTorch {torch.__version__}, CUDA: {torch.cuda.is_available()}, MPS: {torch.backends.mps.is_available()}')"

---

Usage

Web Dashboard (recommended)

Launch the interactive Streamlit UI:

streamlit run ui/app.py

This opens a browser at http://localhost:8501 with:

• Dashboard — pipeline overview and quick-run buttons
• Data Pipeline — upload ELF binaries, run feature extraction
• Visualization — browse generated images (byte plot, entropy, composite RGB)
• Model Training — configure and launch training with live progress
• Evaluation — view confusion matrices, ROC curves, attention maps
• Adversarial — run and visualize adversarial robustness tests
• Live Analysis — upload a single binary for instant classification

CLI: Run the full pipeline

python scripts/run_pipeline.py

CLI: Run in demo mode (no malware samples needed)

Uses benign system binaries to test the full pipeline end-to-end:

python scripts/run_pipeline.py --demo

Run specific steps

# Only feature extraction and visualization
python scripts/run_pipeline.py --steps 1 2

# Only training (assumes images already exist)
python scripts/run_pipeline.py --steps 3 --epochs 30 --batch-size 16

# Only evaluation and adversarial testing
python scripts/run_pipeline.py --steps 4 5

Full CLI reference

usage: run_pipeline.py [-h] [--steps STEPS [STEPS ...]] [--epochs EPOCHS]
                       [--batch-size BATCH_SIZE] [--lr LR]
                       [--device {auto,cpu,cuda,mps}]
                       [--models MODELS [MODELS ...]] [--img-size IMG_SIZE]
                       [--seed SEED] [--verbose] [--demo]

Arguments:
  --steps          Pipeline steps to run (1-5). Default: all.
  --epochs         Training epochs. Default: 50.
  --batch-size     Batch size. Default: 32.
  --lr             Learning rate. Default: 1e-4.
  --device         Training device: auto|cpu|cuda|mps. Default: auto.
  --models         Models to train. Default: vit_tiny resnet18 mobilenetv2.
  --img-size       Image size. Default: 224.
  --seed           Random seed. Default: 42.
  --verbose, -v    Verbose (DEBUG-level) logging.
  --demo           Demo mode using system binaries.

Prepare your own dataset

Place ELF binaries in family-labeled subdirectories:

data/raw/
├── mirai/
│   ├── sample_001.elf
│   └── sample_002.elf
├── gafgyt/
│   └── sample_003.elf
├── tsunami/
│   └── sample_004.elf
└── benign/
    └── /usr/bin/ls  (copy of)

Or use the download helper for guidance:

python scripts/download_samples.py

---

Vendor Repositories

Three open-source repositories are included as reference implementations under vendors/:

Directory	Source	Role in Pipeline
elf-static-analysis/	ashalaginov/Linux-ELF-malware-static-analysis	Reference for Step 1 — ELF feature extraction methodology. Original uses shell commands + MySQL; we replaced with pure-Python pyelftools.
malware-image-detection/	TanayBhadula/malware-image-detection	Reference for Step 2 — binary-to-image conversion. Original handles Windows PE .bytes files; we adapted for raw ELF byte streams.
MalwareViT/	rickyxume/MalwareViT	Reference for Step 3 — ViT model architecture and opcode co-occurrence image generation. We adapted co-occurrence from opcode to byte-level bigrams.

The unified pipeline replaces all vendor dependencies with a consistent PyTorch-based implementation. Vendor code is included for attribution and architectural reference only.

---

Model Details

MalwareViT (DeiT-Tiny variant)

The primary model is a custom Vision Transformer optimized for malware classification:

Input:   224 × 224 × 3  (composite RGB image)
Patches: 16 × 16 → 196 patches
Embed:   192-dimensional patch embeddings + CLS token + positional embeddings
Encoder: 12 Transformer blocks, each with:
         ├── Multi-Head Self-Attention (3 heads, 64 dim/head)
         ├── Layer Normalization
         ├── MLP (192 → 768 → 192)
         └── Dropout (0.1)
Output:  CLS token → Linear → 6 classes

Total Parameters: ~5.7M

The model exposes attention weight extraction via get_attention_maps(), enabling visualization of which binary regions influence classification decisions.

Why ViT over CNN?

• Global receptive field: Self-attention captures long-range dependencies in binary structure (e.g., linking a suspicious import table to unusual section entropy) — CNNs require depth to achieve this.
• Attention interpretability: Attention maps provide built-in explainability, critical for security analysts who need to understand why a binary was flagged.
• Edge efficiency: DeiT-Tiny achieves competitive accuracy at ~5.7M parameters, feasible for Raspberry Pi–class devices.

---

Configuration Reference

All hyperparameters are centralized in src/config.py. Key defaults:

Parameter	Value	Description
image_size	224	Input image dimensions
patch_size	16	ViT patch size
vit_embed_dim	192	Transformer embedding dimension
vit_depth	12	Number of transformer layers
vit_num_heads	3	Attention heads
num_classes	6	Malware families + benign
batch_size	32	Training batch size
num_epochs	50	Maximum training epochs
learning_rate	1e-4	AdamW learning rate
weight_decay	1e-5	L2 regularization
warmup_epochs	5	Linear warmup for LR scheduler
patience	10	Early stopping patience
fgsm_eps	0.03	FGSM attack perturbation budget
pgd_steps	10	PGD attack iterations

Malware families: mirai, gafgyt, tsunami, hajime, bashlite, benign

---

Datasets

This project operates on Linux ELF binaries. No malware samples are distributed with the repository. Researchers can obtain datasets from:

Source	Type	Access
VirusShare	Real malware samples	Free registration
MalwareBazaar	Curated malware database	Open API
IoT-23	IoT malware captures	Free download
EMBER	Labeled PE features	Open (Windows-focused but methodology applicable)
VirusTotal	Hash lookups, family labels	API key

For demo mode, the pipeline extracts benign ELF binaries from the host system (/usr/bin/, /usr/sbin/, etc.) to test the full pipeline without requiring malware samples.

Ethics: This project is intended for defensive security research only. Handle malware samples in isolated environments. Do not execute untrusted binaries outside of controlled sandboxes.

---

References

1. A. Ravi, V. Chaturvedi, M. Shafique, "ViT4Mal: Lightweight Vision Transformer for Malware Detection on Edge Devices," ACM Transactions on Embedded Computing Systems (TECS) / CASES ESWEEK, 2023.
2. A. Ravi, V. Chaturvedi, "Malware Analysis using ELF features for Linux-based IoT devices," IEEE International Conference on VLSI Design (VLSID), 2022.
3. V. Chaturvedi et al., "FRNet: A Feature-Rich CNN Architecture to Defend against Adversarial Attacks," IEEE Access, 2024.
4. V. Chaturvedi et al., "S-E Pipeline: ViT-Based Resilient Classification for Medical Imaging Against Adversarial Attacks," IEEE IJCNN, 2024.
5. A. Dosovitskiy et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," ICLR, 2021.
6. H. Touvron et al., "Training data-efficient image transformers & distillation through attention," ICML, 2021. (DeiT)
7. L. Nataraj et al., "Malware Images: Visualization and Automatic Classification," VizSec, 2011.

---

License

This project is provided for academic and research purposes. Vendor repositories retain their original licenses. See individual vendors/*/LICENSE files for details.