The growth in complexity of statistical methods and the application of those methods to ever-growing datasets by researchers has outpaced the growth in the computing power of uniprocessor computers. Nowhere is this more true than in the biomedical arena, where complex estimation and simulation models have proliferated. Multiprocessor computers are widely available owing to the market for servers, but parallelized statistical software to run on such computers is not commercially available. We propose to extend Stata, a statistical system widely used by researchers, to exploit multiprocessor computers. The goal of the project is to facilitate the estimation of complex statistical models by exploiting the advantages of multiprocessor computers. Our preliminary analysis suggests that Stata on a 2-processor computer can be expected to execute statistical analysis about 1.8 times as fast as a single-processor computer; a 4-processor computer, 3.0 times as fast; a 16 Processor computer, 6.4 times as fast. In the Phase I project, both Stata for Windows and Stata for Linux will be modified and measurements made of the fraction of Stata that can be parallelized and the observed performance improvements. Specific evaluation criteria will then be applied to these measurements.
This project will create a proof-of-concept version of a parallelized edition of the commercially available statistical package Stata. If this proof-of-concept version performs as we expect, we will produce a version for commercial distribution in a subsequent Phase II project. There is no commercial, off-the-shelf statistical package that is parallelized for multiprocessor machines. A commercially available parallelized Stata would serve this growing market.
