The project focuses on the development of interactive help and support systems for novice and expert users of the Student Assessment of their Learning Gains (SALG) course, department, and evaluator websites. The SALG is a web-based, course-evaluation instrument that focuses on students' assessments of how effectively course-design and teaching methods help them make learning gains. The expanded help and support systems increase the usability of the SALG sites for course, program, and departmental evaluation and enhance the power and number of ways that users have to interact with each other.
Improvements to the SALG websites include three types of enhancements. First, the user-support functions expand the help sections on course, department, and evaluator sites by the inclusion of interactive tutorials and "Tips and Tactics" sections. These sections consist of advice about strategies and best practices and an archive of white-papers/articles on the SALG and links to articles on the use of SALG by others. An interactive forum provides a platform for users to access immediate practical help and to engage in discussions on broad issues concerning the assessment of learning. Second, the department site contains additional functions to assist users in exploiting the capabilities of the site by incorporating additional controls and features to assist in departmental level evaluation. Third, in addition to increased functionality to support program evaluation using the SALG, the evaluator site is a repository for SALG-like instruments (e.g., the Undergraduate Research Student Self-Assessment (URSSA) instrument) and includes an interactive forum for sharing information, resources, and expertise on assessment, evaluation, and student learning.
The Student Assessment of their Learning Gains (SALG) website offers valid, reliable, and successful web-based, course-evaluation instruments that focus on students’ assessments of how effectively course-design and teaching methods help them make learning gains. Nearly 10,000 instructors (mostly in STEM disciplines) have active SALG accounts, and draw on the feedback provided by SALG instruments to guide changes to their course designs and pedagogy. This project (NSF Award #0920801) focused on three interlaced objectives: (1) making the website easier to use, especially for novice users; (2) enhancing user support systems; and (3) increasing the size and vitality of the site’s user community, with a long-term goal of making it self-sustaining. At the time of the award, the SALG Development Group was working on two additional SALG websites under NSF Award #0613426. A SALG site for departments was released as a beta version in early 2009, but the proposed site for program evaluators was not completed. Hence, two secondary objectives carried over: (4) to finish developing SALG functions for departments, in particular, to improve usability and help functions on that site; and (5) to develop SALG functionalities for program evaluators. We largely accomplished the first three objectives and made significant progress on the latter two. We implemented a new wizard-based instrument-creation process that made the SALG much easier to use, reducing emails to the help desk from an average of ~50/month to fewer than five. We created a FAQ and instruction sets to guide users through the advanced features of the SALG. Our user community grew extremely rapidly. In 2009, ~1400 faculty had created SALG accounts, 900 surveys had been completed, and 21,000 students had been surveyed. Currently, the SALG site shows 9905 instructor users, 9268 completed surveys and 214,223 student responses. As of mid-2011, 181 users had created ~1860 department instruments, administered over 260 surveys, and collected data from ~3500 students. Currently, 4147 SALG departments have created 21,738 department instruments, 712 surveys have been conducted and over 199,000 students have responded to those department surveys. Our user community is not only growing, the growth curve is accelerating. We started a user group, and conducted an extensive needs analysis that helped us better understand why people adopt the SALG and how they use it. We also formed collaborative relationships with several large STEM-education reform groups and with the Western Association of Schools and Colleges. We are working with two universities piloting the use of SALG as an institution-wide assessment instrument. We also developed Student Assessment of their Improvement in Learning (SAIL), a paper-and-pencil assessment instrument based on SALG principles, which has proved effective in short-duration and informal educational settings. Extensive testing allowed us to develop help systems for the department site and reshaped our upcoming evaluator functions. Overall, we are pleased with what we have achieved so far. As we developed the department site however, we discovered that the present SALG database cannot support the analytical and reporting requirements of the department or evaluator sites because it offers no provisions for aggregating data across groups. It also doesn’t enforce the data integrity necessary for data-mining, severely compromising its usefulness as a research tool. The SALG site was originally designed as a way for individual instructors to get feedback on their course designs and pedagogy, so provisions for group functions were not built into the architecture. Moreover, as SALG gained functions, resulting complexities in the programming and database made it increasingly expensive to maintain and upgrade. We therefore decided to rebuild. We designed SALG2 from the outset to meet present and anticipated needs of instructors, departments (and other groups), program evaluators and researchers in a single, integrated site, reducing expense and increasing ease of use. Taking full advantage of all we learned over the past 8 years, we eliminated all known bugs and annoyances and added several long-sought features (e.g., ability to drag and drop questions). We also built in capacity for new features such as dashboards and control panels that allow group administrators to monitor open surveys and progress toward goals. SALG2 integrates social-media-based user support systems directly into the website. The new site is faster, more powerful, more stable and reliable, and more scalable than the previous site. SALG2 is now an operational (but not yet public) website. Users can create, modify, deliver and analyze SALG surveys on the new site. Beta-testers reported that SALG2 is significantly easier to use than the old site and appreciated the new features and faster performance. We anticipate that continuing improvements to the site will lead not only to greater uptake among first-time users, but that they will enhance commitment, vitality and interactivity among SALG users, making the site more self-supporting. We have applied for additional funding to complete the transition to and further development of SALG2.
