Executive Summary : | Online education is becoming very popular these days. It makes the courses accessible to a large number of students and allows their interactions with the course content/facilitators to be digitized for further use to enhance the learning experience. This project involves building a smart music tutor (SMT) for Indian classical music which can be used by music students, ranging from beginners to advanced levels of expertise. To students, it will play the teacher’s audio (vocal or instrumental music) lessons, and give lessons to practice. While practicing assignments, the students’ audio will be recorded, and further, analyzed to assess their performance, detect mistakes, and give them constructive feedback. The performance of a student will also decide her next set of lessons and/or practice exercises. To the teachers, SMT will be an assistant to enhance their outreach by providing them with smart tools to plan and conduct their courses. These tools will be based on machine learning (ML) and signal to process and will include automatic lesson delivery tailored for each student (based on her performance and interest), automatic assessment of student's audio, analyzing anomalous behavior, detecting strengths and weaknesses of students, and adapting the ML models based on the user (teacher or student) interaction. The project will involve conceptualizing the overall system and developing technologies for its various features. Investigators already have developed some algorithms to start with - like melody estimation and polyphonic music transcription using matrix factorization methods. In the first phase of this project, they will be developing a preliminary version for a selected set of students and teachers. Based on the data they collect by their interaction with this prototype system, investigators will carry out further research and development. Investigators would like to implement automatic methods for music transcription (converting audio to musical notations), melody extraction (estimating the main sequence of notes), metrics for music quality (to assess the performance of students and find mistakes), personalized lesson planning (to recommend practice lessons to students based on their mistakes and strengths), music search (to find similar audio pieces for lesson recommendation), active learning (selecting student’s pieces which need teacher’s attention), and semi-supervised learning (to adapt models based on the feedback from teachers). This project will help not only in popularizing and preserving various forms of Indian classical music but will also create experts by making the lessons available at everyone’s fingertips. Moreover, it will help the maestros by enhancing their teaching outreach. In summary, this project will be developing a smart music teacher app as a product, while expanding research into several problems associated with this product. |