IndoFineTune

Multi-agent fine-tuning pipeline for Indonesian LLMs, optimized for AMD Instinct MI300X with ROCm.

Why IndoFineTune?

Fine-tuning Indonesian LLMs is hard. You need:

• Clean, high-quality training data
• Proper LoRA/QLoRA configuration for local hardware
• Evaluation that actually measures Indonesian language quality
• Deployment that works on your GPU

IndoFineTune automates this with 4 specialized AI agents that coordinate end-to-end:

📦 DataPrep → 🏋️ Trainer → 📊 Evaluator → 🚀 Deployer

Each agent is stateless, typed (Pydantic), and produces an audit trail for full reproducibility.

Architecture

┌─────────────────────────────────────────────────────────┐
│                    Orchestrator                          │
│              (Rich progress + audit trail)               │
└─────────────────────────────────────────────────────────┘
        │            │            │            │
        ▼            ▼            ▼            ▼
┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐
│ DataPrep │  │ Trainer  │  │Evaluator │  │ Deployer │
│          │  │          │  │          │  │          │
│ • Clean  │  │ • LoRA   │  │ • Loss   │  │ • vLLM   │
│ • Augment│  │ • QLoRA  │  │ • BLEU   │  │ • TGI    │
│ • Split  │  │ • ROCm   │  │ • ROUGE  │  │ • ROCm   │
└──────────┘  └──────────┘  └──────────┘  └──────────┘

Quick Start

# 1. Clone & install
git clone https://github.com/teguh407/indofinetune.git
cd indofinetune
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt

# 2. Configure
cp .env.example .env
# Edit .env with your API keys and model path

# 3. Run
python -m src.main --dataset data/train.jsonl --augment

Agent Pipeline

📦 DataPrep Agent

• Loads Alpaca/ShareGPT/custom formats
• Deduplication, quality filtering
• LLM-powered data augmentation (optional)
• Train/val/test splitting

🏋️ Trainer Agent

• LoRA/QLoRA configuration
• ROCm-compatible device detection (AMD MI300X)
• WandB logging
• Checkpoint management

📊 Evaluator Agent

• Perplexity, loss, BLEU, ROUGE
• Side-by-side baseline comparison
• Deployment recommendation

🚀 Deployer Agent

• vLLM server launch
• ROCm environment setup
• API endpoint generation

AMD Instinct MI300X Optimization

IndoFineTune is designed for AMD's flagship data center GPU:

Spec	MI300X Value
Memory	192 GB HBM3
Memory Bandwidth	5.3 TB/s
FP16 Performance	1,307 TFLOPS
ROCm Support	7.2+

ROCm Setup

# Install ROCm 7.2+
# https://rocm.docs.amd.com/en/latest/

# Verify
rocminfo | head -20

CLI Reference

# Basic fine-tuning
python -m src.main --dataset data/train.jsonl

# With data augmentation
python -m src.main --dataset data/train.jsonl --augment

# Custom model & hyperparameters
python -m src.main --dataset data/train.jsonl \
  --base-model meta-llama/Llama-3-8B \
  --epochs 5 \
  --lr 1e-5 \
  --batch-size 8

# Skip deployment
python -m src.main --dataset data/train.jsonl --skip-deploy

Project Structure

indofinetune/
├── README.md
├── pyproject.toml
├── requirements.txt
├── .env.example
├── src/
│   ├── __init__.py
│   ├── config.py           # Pydantic-Settings loader
│   ├── llm.py              # OpenAI-compatible LLM client
│   ├── schemas.py          # Pydantic typed contracts
│   ├── orchestrator.py     # Pipeline orchestration
│   ├── main.py             # CLI entry point
│   └── agents/
│       ├── __init__.py     # BaseAgent
│       ├── data_prep.py    # Data preparation
│       ├── trainer.py      # LoRA/QLoRA training
│       ├── evaluator.py    # Model evaluation
│       └── deployer.py     # Model deployment
├── tests/
│   └── test_smoke.py       # Deterministic smoke tests
└── docs/
    └── index.html          # GitHub Pages landing

Roadmap

• v0.2: Multi-GPU training support (tensor parallel)
• v0.3: Indonesian evaluation benchmarks (ID-BLEU, ID-ROUGE)
• v0.4: Web UI with training dashboard
• v0.5: Integration with HuggingFace Hub for model sharing
• v1.0: Production-ready with auto-scaling deployment

License

MIT © Teguh