The user interfaces (UIs) of a web application serve an important role in the success of online companies. Studies have shown that the aesthetics of an application?s UIs can strongly influence users? overall evaluation of a company; in particular impressions of trustworthiness and usability. Presentation failures ? a discrepancy between the actual appearance of a web site and its intended appearance ? can undermine this effort and negatively impact end users? perception of the quality of the site, the services it delivers, and affect the branding a company is trying to achieve. Debugging presentation failures is typically a manual task where testers must identify when the layout and style of an app?s pages is incorrect and developers must carry out repairs. This process is both labor intensive and error prone: testers can easily miss presentation failures, since the process is based on the testers? visual comparison of a page and an oracle; and it can be difficult for developers to identify the responsible HTML elements or properties once a failure is detected, since HTML and CSS interact in complex ways. In this research effort the PIs will investigate automated techniques to help testers more accurately detect presentation failures and efficiently identify the faulty code. Given the widespread deployment of web applications, this work will impact both end users and software developers by reducing errors in these applications and making the debugging process more effective and efficient. The results of this research will also impact educational efforts through the training of future software engineering in new techniques to improve the quality of web applications.
The technical research will encompass two general areas. The first area of research will advance techniques for detecting presentation failures. The primary mechanism for doing this will be to apply computer vision techniques to automatically identify discrepancies between the actual rendered appearance of a UI and its intended appearance. The developed techniques will be widely applicable to a range of UIs, including those with dynamic content and those using different specification mechanisms for UI oracles. The second area of research will use the identified differences to identify the specific elements in the UI code that are responsible for the observed failures. The research efforts will focus on using artificial intelligence techniques, such as search-based and statistical-based learning, on the UI?s layout models to reason about the UI?s possible faults and guide the developers to identify the faulty HTML elements and CSS properties.
