CTFN in inspired by CITOFONO (Doorbell in Italian) the only instrument every human played at least once
Apple Silicon / MPS fork of HeartMuLa Studio
AI music generation studio for HeartLib, optimized for Apple Metal (M-Series & Intel Macs)
Features • Installation • Usage • Configuration • Credits
This repository is a fork of HeartMuLa Studio (rebranded as CTFN Studio) focused on Apple MPS (Metal Performance Shaders)
- Native macOS app — Standalone
.app(pywebview/Cocoa) - MPS-accelerated inference — HeartMuLa and HeartCodec on Metal GPUs
- Unified Memory friendly — 4-bit quantization and MPS fallbacks
| Feature | Description |
|---|---|
| Full Song Generation | Create complete songs with vocals and lyrics up to 4+ minutes |
| Instrumental Mode | Generate instrumental tracks without vocals |
| Style Tags | Define genre, mood, tempo, and instrumentation |
| Seed Control | Reproduce exact generations for consistency |
| Queue System | Queue multiple generations and process them sequentially |
| Feature | Description |
|---|---|
| Audio Upload | Use any audio file as a style reference |
| Waveform Visualization | Waveform display powered by WaveSurfer.js |
| Region Selection | Draggable 10-second region selector for precise style sampling |
| Style Influence | Adjustable slider to control reference audio influence (1–100%) |
| Synced Playback | Modal waveform syncs with bottom player in real time |
| Feature | Description |
|---|---|
| Lyrics Generation | Generate lyrics from a topic using LLMs |
| Multiple Providers | Support for Ollama (local) and OpenRouter (cloud) |
| Style Suggestions | AI-suggested style tags based on your concept |
| Prompt Enhancement | Improve your prompts with AI assistance |
| Feature | Description |
|---|---|
| Modern UI | Clean layout with dark/light mode |
| Bottom Player | Player with waveform, volume, and progress |
| History Feed | Browse, search, and manage generated tracks |
| Likes & Playlists | Organize favorites into custom playlists |
| Real-time Progress | Live generation progress with step indicators |
| Responsive Design | Works on desktop and mobile devices |
| Layer | Technologies |
|---|---|
| Frontend | React 18, TypeScript, TailwindCSS, Framer Motion, WaveSurfer.js |
| Backend | FastAPI, SQLModel, SSE (Server-Sent Events) |
| AI Engine | HeartLib — MuQ, MuLan, HeartCodec |
| LLM Integration | Ollama, OpenRouter |
| Acceleration | PyTorch MPS (Apple Metal) |
This fork is tuned for Apple Silicon (M1/M2/M3) and Intel Macs with Metal:
- MPS device — HeartMuLa and HeartCodec run on MPS when available; CPU fallback for unsupported ops.
- 4-bit quantization — Lowers memory use (e.g. ~11GB → ~3GB equivalent) for Unified Memory:
HEARTMULA_4BIT=true python -m uvicorn backend.app.main:app --host 0.0.0.0 --port 8000
- MPS decode — Audio decoding uses Metal where possible; runtime patches for heartlib compatibility are documented in MPS_OPTIMIZATION.md.
Models are auto-downloaded from Hugging Face on first run (~5GB): HeartMuLa, HeartCodec, tokenizer, and config.
- Download the latest CTFNStudio-macOS.dmg (or
.zip) from Releases. - Open the DMG and drag CTFN Studio.app to Applications.
- Double-click to launch. If macOS blocks it, use Right-click → Open, or run:
xattr -cr /Applications/CTFNStudio.app
Data is stored in your user Library, not inside the app:
~/Library/Application Support/HeartMuLa/
├── models/ # AI models (~5GB, auto-downloaded)
├── generated_audio/ # Generated music files
├── ref_audio/ # Uploaded reference audio
└── jobs.db # Song history
~/Library/Logs/HeartMuLa/
└── (logs)
Requirements: macOS 10.13+, Apple Silicon or Intel Mac with Metal, 10GB+ RAM, ~15GB free disk.
Prerequisites: Python 3.10+, Node.js 18+, Git.
git clone https://github.com/audiohacking/HeartMuLa-Studio.git
cd HeartMuLa-StudioBackend:
python -m venv venv
source venv/bin/activate
pip install -r backend/requirements.txtFrontend:
cd frontend
npm install
npm run build
cd ..Quick start (dev):
./start.shThen open http://localhost:5173 (dev) or http://localhost:8000 (production).
Backend only:
source venv/bin/activate
python -m uvicorn backend.app.main:app --host 0.0.0.0 --port 8000Frontend dev:
cd frontend && npm run devMPS is used automatically when available; no extra flags needed.
Create a .env file in backend/ for optional settings:
OPENROUTER_API_KEY=your_api_key_here # Cloud LLM
OLLAMA_HOST=http://localhost:11434 # Local LLMBackend / launcher env (optional):
| Variable | Default | Description |
|---|---|---|
HEARTMULA_MODEL_DIR |
backend/models or App Support |
Model directory |
HEARTMULA_4BIT |
auto |
4-bit quantization: auto, true, false |
HEARTMULA_SEQUENTIAL_OFFLOAD |
auto |
Model swapping for low RAM: auto, true, false |
HEARTMULA_VERSION |
RL-3B-20260123 |
Model version |
| Method | Endpoint | Description |
|---|---|---|
POST |
/generate/music |
Start music generation |
POST |
/generate/lyrics |
Generate lyrics (LLM) |
POST |
/upload/ref_audio |
Upload reference audio |
GET |
/history |
Generation history |
GET |
/jobs/{id} |
Job status |
GET |
/events |
SSE stream for progress |
GET |
/audio/{path} |
Stream generated audio |
| Issue | Suggestion |
|---|---|
| Models not downloading | Check internet and disk space (~5GB in backend/models or App Support) |
| Frontend can’t connect | Ensure backend is running on port 8000 |
| LLM not working | Run Ollama locally or set OpenRouter API key in backend/.env |
| Slow or high memory | Try HEARTMULA_4BIT=true or HEARTMULA_SEQUENTIAL_OFFLOAD=true |
| macOS app won’t open | Use Right-click → Open, or xattr -cr /Applications/CTFNStudio.app |
./local_build.shThis installs deps, builds the frontend, generates the icon, runs PyInstaller, and signs the bundle. Output: dist/CTFNStudio.app. See build/macos/README.md and .github/workflows/build-macos-release.yml for details.
- HeartMuLa/heartlib — Open-source AI music generation engine used by this project.
- WaveSurfer.js — Audio waveform visualization.
CTFN Studio is maintained for Apple MPS / Metal. The underlying AI engine is HeartMuLa/heartlib; this project is not affiliated with the upstream HeartMuLa project beyond use of HeartLib.