How can a computer learn to read hidden text in ancient manuscripts? How can a computer be trained to transcribe music audio into music notation or compose a new song? Can a computer system teach a person to better use prosody — the musical pattern of speech – to become a more effective public speaker? These are some of the questions that students will investigate in this Research Experience for Undergraduates (REU) site at the University of Rochester. Students will explore an exciting, interdisciplinary research area that combines computer science, electrical engineering, cognitive science, and music. Each student will be mentored by two or more faculty members from the University's schools of engineering, sciences, arts, and music. Other activities of the REU site include workshops on career development; scholarship community colloquiums; graduate school preparation; Python programming for machine learning; and music-focused activities. The goals of this REU are to increase the diversity and broaden the horizons of students engaged in computer science research. The themes of music, digital media, and cognitive science will attract many students from groups under-represented in computer science. The site welcomes students from institutions where opportunities for interdisciplinary research combining computational methods, arts and humanities may be limited. Students who are already majoring in computer science will discover that the research in the field is not limited to traditional engineering applications, but can address questions of art, culture, and human psychology. Students with experience in combining computer science with humanistic research are already in great demand in industry and academia and will help define what it means to be a computer scientist in the 21st century.
Students in this REU will engage in interdisciplinary research that combines machine learning, computer audition, music theory, and cognitive science. These disciplines are united by their use of a common set of formal representations and computational methods; in particular, probabilistic models and machine learning. In the research activities, REU students will work on a variety of topics. These include: creating human-computer collaborative music making systems; developing electroencephalogram (EEG) analysis methods to decode musical minds; using machine learning and multispectral imaging to read hidden text in ancient manuscripts; studying how prosody makes a person a convincing public speaker; developing big data techniques to analyze audio-visual scenes; and, exploring connections between human and computer natural language processing. The program is strengthened by the University's recently-founded audio and music engineering program, and the partnership with the Eastman School of Music. Although the major objective of the REU is to encourage students to enter STEM graduate programs, many of the projects can be expected to lead to novel and publishable research.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
