PianoAR
AR application for teaching playing piano
Meet Our Team
Gal Shalom, Ariel Iny, Alex Bondar
Bachelor Computer Science Students, Technion ,Institute of Technology, Israel
Goals
Use an AR application to teach how to play piano.
Using Unity as a platform for developing MR,
and Microsoft Hololens v1 as a AR device
Primary Objectives
Easy to use
Accurate piano recognition
Lite weight key-press detection
Positive feedback for playing
Intuitive UI
Features
Piano Keyboard Detection
Voice Command Interface
Key Press Detection
Multiple songs to learn
Interactive feedback
Useable on any piano
Piano Recognition
Image Processing recognition
The recognition of the piano keyboard using OpenCV library, using linear operations of image processing.
After that, Placing virtual piano model on top the real piano.
Challenges:
Learning image processing
Build Piano 3D model in Unity
Recognition of piano keyboard, without any assumptions on the piano
Transforming 2D coordinates to 3D coordinates
Light resource algorithm for piano recognition
Hololens weak processing power
Keyboard Calibration
Keys Calibration
Under the assumption, the user won’t move this head.
We calibrate the middle octave of the keyboard to be detected by the application when pressed by the used.
Challenges:
Learning image processing
Calibration and detection method while keeping it as simple as possible.
Few limitations as possible on the user
Hololens weak processing power
Finger Recognition
Demo
Development Process
