Abstract

Web sites, such as Amazon.com and Travelocity, have become an integral way to perform daily operations in business, government, and academia. Since customers depend on these online sites, the correctness of the web applications driving them is crucial. The critical need for reliable web applications motivates the development of inexpensive, effective testing methods.

One cost-effective approach to testing web applications is capture/replay testing, where a tester (1) creates tests based on real user requests (also known as "field data'') to the deployed application, (2) automatically replays those tests, and (3) evaluates the outcomes of the executed tests using an oracle. Current approaches to each of the three steps have limitations, and no comprehensive testing framework that allows testers to apply various strategies to each step exists.

To address the shortcomings of current approaches, this dissertation explores alternative automated testing strategies, which are implemented and integrated into a customizable web application testing framework, to expose faults effectively. The main thesis behind this research is that the practical effectiveness of automatically exposing faults in web applications can be significantly improved through an integrated strategy of test-case generation, replay, and oracle definition and comparison algorithms customized to web applications.

This dissertation presents test-case generation, replay, and oracle components that can be integrated into a customizable, automated framework for testing web-based software. Manually examining and evaluating the test output without a means for viewing the test results is both tedious and error-prone. To support analyzing the test output from evaluating the components, I have designed and implemented WEBVIZOR, (WEB application fault detection VIsualiZation with ORacles), an open-source tool that supports testers in analyzing test case results for web applications. Based on the results of empirical studies comparing the components, I make recommendations for the test-case generation strategy, replay mechanism, and oracle comparator that meet a tester's fault exposure goals within some resource constraints. Furthermore, I applied the customizable testing framework to perform an empirical comparison of test-suite reduction techniques. The impact of this dissertation is improved reliability of web applications using automated techniques that will not significantly burden testers.