Although the benefits of note-taking in the classroom are widely recognized, there has not been enough research focused on alleviating the difficulties encountered by legally blind and low vision students in their attempts to take notes during lecture. The problem is particularly acute in fast-paced STEM courses. Students who are legally blind typically write by placing their head close to the writing surface. They may be able to use a monocular to see what is being written on a board in the front of the classroom. But monoculars with high magnification also have narrow fields of view, which forces the student to "hunt" for the target at the front of the classroom each time s/he looks up from the writing surface. The repeated delay in switching between the writing surface and the board can make it hard for the student to keep up. In this project, the PI will develop and evaluate a portable Note-Taker device that does not require any adaptation of the existing classroom infrastructure, and which allows visually impaired students to shift their attention between the writing surface and the class presentation without inefficient context switching. The device will employ a Tablet PC, a zooming video camera, and an electronic pan/tilt mechanism, which can all be easily carried in a backpack and set up in a few seconds on any classroom desk. On the Tablet PC's display surface the student will be able both to see a zoomed video of the lecturer's presentation at the front of the classroom in real time, and to take notes with digital ink. The student will be able to adjust the camera's aim at any time by simply tapping on the point of interest in the video window on the display surface of the Tablet PC. The PI's goal is to go beyond mere "accessibility" and to create a device that allows legally blind students to take notes as efficiently as fully sighted students. Development and evaluation of the Note-Taker prototype will be done with the full involvement of legally blind and low vision students on the campus of Arizona State University, under the auspices of the Cognitive Ubiquitous Computing Center for Assistive and Rehabilitative Systems (CUbiC CAReS). The PI hypothesizes that his Note-Taking device will improve the learning of students who employ it in their secondary or post-secondary classrooms to take notes during lectures, and that it will also help those students to review their own notes at a later time, in conjunction with cross-referenced audio and video recordings.
Broader Impacts: Difficulties in note-taking are not limited to students with low vision. Students with certain learning disabilities, for example, often also have difficulty taking notes at the pace at which material is presented in the classroom. The PI's Note-Taker will reduce irrelevant stimuli, thereby making it easier for such students to successfully absorb, record, and ultimately understand the material presented in the classroom.
The problem we aimed to solve It's often difficult for students who are visually impaired to see whiteboards, chalkboards, and PowerPoint slides at the front of their classrooms. The objective of the Note-Taker project was to develop a portable device that these students could carry with them to class and set up on their desktop, to provide a zoomed-in view of the front of the classroom. But just seeing is not enough. Students also need to take notes. Classroom note-taking helps students recall information – even if they never look at their notes after class. That's because Note-taking forces a student to think about what's being taught. So we wanted to also make a device that would help students take notes in class. The guiding design principles for the Note-Taker Based on these important problems, the Note-Taker project team agreed on nine principles to guide their design of their Note-Taker device: 1. The Note-Taker should not rely upon the presence of previously installed equipment in the classroom. 2. The Note-Taker should not make users dependent on others, including other students, the instructor, or a member of the technical staff. 3. The Note-Taker should not interfere with the student's classroom discussions with instructors or fellow students, as might be the case with a head-mounted camera system. 4. The Note-Taker should not require teachers to change their presentations in any way. Ideally, the teacher should not even know that a special device is being used by the student. 5. The Note-Taker should not disrupt the classroom. For example, it should not block the views of other students, and it should not be noisy. 6. The Note-Taker should be reasonably priced. 7. The Note-Taker should allow the student to see everything important, including the teacher, the whiteboard, and any PowerPoint slides. 8. The Note-Taker should be portable, it should be very easy to set up, and it should be small enough to fit on the student’s classroom desk or table. 9. The Note-Taker should allow students who are visually impaired to see the presentation at the front of the class while it is happening, just as well as the fully sighted students. The Note-Taker prototypes During the course of the Note-Taker project, the research team developed four Note-Taker prototypes. After building each prototype, they had students with visual disabilities use it, and give them feedback. That allowed them to improve their design with each new prototype. The 1st Generation Note-Taker prototype was called the "proof of concept" prototype. It used an off-the-shelf pan/tilt mechanism that was rather wobbly. Thankfully, it worked well enough to help David (a student on the Note-Taker team who is legally blind) get through his final 3 senior math classes, and complete his Math degree. It also helped convince a review panel at the National Science Foundation to fund further work on the Note-Taker project. Using the NSF funding, the Note-Taker team then built their 2nd Generation prototype, which they called the "SkippyCam" Note-Taker, because its video camera was housed in a Skippy peanut butter jar. The 2nd Generation Note-Taker prototype was still rather fragile, but it was more reliable. It was also good enough to win First Prize in the 2010 Microsoft Imagine Cup World Competition, under the "Touch and Tablet Accessibility" category. Based on feedback from the students who used the 2nd Generation Note-Taker, the project team then developed a 3rd Generation Note-Taker. It was much more rugged, and it aimed the camera much more precisely. The 3rd Generation Note-Taker won a second place Silver Award in the student section of the International Design Excellence Awards (IDEA) Competition. It also won First Place in the 2011 Microsoft Imagine Cup US Competition, and Second Place in the Microsoft Imagine Cup World Competition. Ira Flatow then interviewed one of the Note-Taker team members on NPR’s Science Friday program. While the students who tested the 3rd Generation Note-Taker prototype liked its features, they said that it was just too big and heavy to carry from class to class in a backpack. To reduce the size and weight, the Note-Taker team looked for a way to get rid of the pan and tilt servo motors. They accomplished this by substituting a newly introduced higher-resolution USB 2.0 digital video camera, which allowed them to do panning and tilting digitally. The resulting 4th Generation Note-Taker prototype was described as "simple and highly portable". Current Note-Taker deployment The students who tested the 4th Generation Note-Taker prototype really liked it. A total of 12 of these prototypes were eventually built, and those prototypes are currently being provided to students on four Arizona university campuses: Arizona State University, Arizona State PolyTechnic, Northern Arizona University, and University of Arizona. That same 4th Generation Note-Taker is also now being offered to the public by Essistive Technologies.
