Harmosee

Table of Contents

• Abstract
  
• Requirements
  
• Installation and usage
  
• Preview
  
• Team
  
• Contribution
  
• Improvements

Abstract

The Harmosee application demystifies musical notation, translating complex sheet music to a clear visual guide by providing a virtual keyboard highlighting the respective keys. The application streamlines the learning process of music instruments, mitigating the challenges of interpreting sheet music. The application can be a vital tool for the visualisation of music notations and their respective representations on an instrument.

Requirements

Language/Library	Version/Link
Python	3.10+
OpenCV	4.9+
Music21	7.3+
FluidSynth	2.3+
NumPy	1.26+
Streamlit	1.32+
Roboflow Model API	(Used for note detection and symbol classification)
SoundFont	General MIDI (.sf3)

Installation and usage

To get this project up and running on your local machine, follow these steps.

1. Clone the repository

   git clone https://github.com/AAC-Open-Source-Pool/25AACL04.git

2. Install System Dependencies This project requires FluidSynth. Please install it using your system's package manager.

   • On Ubuntu/Debian: sudo apt-get install fluidsynth
   • On macOS (using Homebrew): brew install fluidsynth
   • On Windows: Download and install from the official FluidSynth website.

3. Download a SoundFont 🎵 FluidSynth needs a SoundFont file (.sf2 or .sf3) to generate audio.

   • Download a general-purpose SoundFont. We recommend the one linked in the Requirements table above.
   • Place the downloaded file inside your project's main folder.

4. Create a Virtual Environment (Recommended) Navigate into your project directory and run the following commands:

   python -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`

5. Install Python Packages Install all the required packages from the requirements.txt file.

   pip install -r requirements.txt

6. Run the Application Launch the Streamlit web application. Make sure to replace app.py with the actual name of your main Python file.

   streamlit run app.py

7. How to Use

   • Open the web application in your browser (the URL will appear in your terminal).
   • Upload an image of the sheet music you want to visualize.
   • The application will preprocess the image, detect notes and clefs, and generate an output video.
   • The final output is a video of a virtual piano playing the music with the corresponding keys highlighted.

Preview

Here are some screenshots of the Harmosee application in action.

Team details

Team Number:

25AACCL04

Senior Mentor:

Sujay

Junior Mentor:

Sruthi

Team Member 1:

Gummadi Joshita

Team Member 2:

Pabba Meghana

Team Member 3:

Asmitha Veluru

Team Member 4:

Koritala Yamini

Contribution

This section provides instructions and details on how to submit a contribution via a pull request. It is important to follow these guidelines to make sure your pull request is accepted.

1. Before choosing to propose changes to this project, it is advisable to go through the readme.md file of the project to get the philosophy and the motive that went behind this project. The pull request should align with the philosophy and the motive of the original poster of this project.
2. To add your changes, make sure that the programming language in which you are proposing the changes should be the same as the programming language that has been used in the project. The versions of the programming language and the libraries(if any) used should also match with the original code.
3. Write a documentation on the changes that you are proposing. The documentation should include the problems you have noticed in the code(if any), the changes you would like to propose, the reason for these changes, and sample test cases. Remember that the topics in the documentation are strictly not limited to the topics aforementioned, but are just an inclusion.
4. Submit a pull request via Git etiquettes

Improvements

Here are some potential features and enhancements that could be added to Harmosee in the future:

• Improve Model Accuracy: Train the note detection model on a larger and more diverse dataset of sheet music to handle different styles, fonts, and low-quality images more effectively.
• Support for More Instruments: Add visualization for other instruments like guitar (fretboard), violin, etc., in addition to the piano.
• Real-time Processing: Implement functionality to process sheet music in real-time using a device's camera instead of static image uploads.
• PDF and Multi-page Support: Allow users to upload multi-page PDF files, not just single images, and process them sequentially.
• Interactive Playback Controls: Add controls to the video player to pause, rewind, change the tempo of the playback, and loop specific sections for practice.
• Support for More Musical Symbols: Enhance the model to recognize a wider range of musical notations, such as dynamics (e.g., forte, piano), articulations (e.g., staccato, legato), and ornaments.