🎶 Virtual MIDI Hands – Touchless Music Creation

Play Any Instrument Using Only Your Hands and a Webcam

🎥 Demo

Click below to watch the demo video:

---

✨ What is Virtual MIDI Hands?

A webcam-powered, real-time virtual musical instrument that transforms your hand gestures into MIDI chords and notes.
No physical instrument needed — your fingers are the keys.

• Detects hand gestures via webcam
• Maps gestures to MIDI chords from the C Major / A Minor scale
• Works with any MIDI-compatible instrument — piano, guitar, synth, drums, etc.
• Sends real-time MIDI signals with automatic note sustain (2 seconds)

---

💻 Tech Stack

Tool	Purpose
OpenCV	Live webcam feed & image processing
MediaPipe	High-accuracy hand tracking engine
cvzone	Simplified wrapper for hand detection
pygame.midi	MIDI signal generation
threading	Smooth non-blocking note sustain

---

🚀 Getting Started

1. Clone the Repository

git clone https://github.com/kaleprabhat24/MidiHands.git
cd MidiHands

2. Install Dependencies

Ensure Python 3.7 or higher is installed.

pip install -r requirements.txt

If cvzone doesn't install correctly:

pip install git+https://github.com/cvzone/cvzone.git

---

🎛️ How It Works

1. Webcam captures live video.
2. MediaPipe locates hands and fingers.
3. Raised fingers are mapped to specific chords.
4. pygame.midi plays MIDI notes in real-time.
5. Notes automatically stop after a 2-second sustain.

---

🎼 Chord Mapping – C Major / A Minor Scale

Left Hand:

Finger	Chord	Notes (MIDI)
Thumb	C Major	60, 64, 67
Index	D Minor	62, 65, 69
Middle	E Minor	64, 67, 71
Ring	F Major	65, 69, 72
Pinky	G Major	67, 71, 74

Right Hand:

Finger	Chord	Notes (MIDI)
Thumb	A Minor	69, 72, 76
Index	C Major	60, 64, 67
Middle	D Minor	62, 65, 69
Ring	E Minor	64, 67, 71
Pinky	F Major	65, 69, 72

---

🧠 Behind the Scenes

• cvzone.HandDetector() tracks hands and fingers.
• fingersUp() checks which fingers are raised.
• Each finger triggers note_on; sustain managed by Python threads.
• note_off is delayed by 2 seconds using threading.Timer.

---

🎮 Features

• 👋 Real-time hand gesture control — touchless & intuitive
• 🎹 Use any virtual instrument (FL Studio, Synths, etc.)
• 🧩 Beginner-friendly code — easily customizable
• 🔄 Scalable — add more gestures, notes, or instruments
• 🛠️ Works with both hands simultaneously

---

🎨 Customization Options

• 🎻 Change Instrument:

  player.set_instrument(<instrument_id>)

  Refer to General MIDI Instrument List

• 🔁 Change Scale or Chords: Edit the chord_map dictionary to assign new chords or keys.

• ⏱️ Modify Sustain Duration: Change the value of SUSTAIN_TIME (default: 2.0 seconds).

---

📦 Requirements

absl-py==2.2.2
attrs==25.3.0
cffi==1.17.1
contourpy==1.3.2
cvzone==1.6.1
cycler==0.12.1
flatbuffers==25.2.10
fonttools==4.57.0
jax==0.6.0
jaxlib==0.6.0
kiwisolver==1.4.8
matplotlib==3.10.1
mediapipe==0.10.21
ml_dtypes==0.5.1
numpy==1.26.4
opencv-contrib-python==4.11.0.86
opencv-python==4.11.0.86
opt_einsum==3.4.0
packaging==24.2
pillow==11.2.1
protobuf==4.25.6
pycparser==2.22
pygame==2.6.1
pyparsing==3.2.3
python-dateutil==2.9.0.post0
scipy==1.15.2
sentencepiece==0.2.0
six==1.17.0
sounddevice==0.5.1

Install them using:

pip install -r requirements.txt

---

💡 Use Cases

• 🎵 Beginners exploring music with no hardware
• ♿ Touchless option for disabled musicians
• 🎓 Ideal for student projects, educators, or hobbyists
• 🎧 Connect directly to FL Studio or any DAW via virtual MIDI ports

---

📘 Conclusion

A creative fusion of AI + Computer Vision + Music —
This project shows how simple gestures can become powerful, expressive tools for creating sound.
Low-cost, fun, and full of potential.

---

🤝 Credits

• cvzone by Murtaza Hassan
• pygame.midi for MIDI integration
• OpenCV and MediaPipe for computer vision tracking

---