This project pursues three different projects on multiple testing: 1. Consider the problem of testing s null hypotheses simultaneously. The classical approach to such a problem is to require control of the Familywise Error Rate (FWER), the probability of even one false rejection. Unfortunately, when s is large, the ability of such a procedure to detect false null hypotheses may be very limited. For this reason, it is often preferred in such situations to relax control of the FWER. In joint work with Joseph Romano and Michael Wolf, this project develops methods for this problem that asymptotically control the false discovery rate (FDR), the expected value of the fraction of rejections that are false rejections (defined to be 0 in the case of no rejections). Unlike existing methods for control of the FDR, the methods developed by this project incorporate information about the joint distribution of the test statistics when determining which null hypotheses to reject and, thus, are better able to detect false null hypotheses. The project illustrates this property via a simulation study and an application to the evaluation of hedge funds. 2. The literature on program evaluation typically focuses on estimation of the average effect of a program for all individuals or for all treated individuals. Of course, even if these quantities are zero, it may be the case that the average effect conditional on some value of observed covariates is nonzero. Identifying the values of the covariates for which this is true may be of substantive interest, especially to policy makers interested in extending the program or treatment to other populations. 3. Randomized trials of programs in developing countries have become increasingly popular within development economics. In many cases, the program involves a single treatment, but there are multiple outcomes of interest. A prominent example of this is PROGRESA, a large-scale, on-going poverty reduction program in Mexico started in 1998. Existing studies have found that PROGRESA has an effect on a large number of different outcomes when each outcome is considered individually. Many of these findings may, however, be due to false rejections, leading to an overstatement of the effect of the program. Together with Soohyung Lee and Joanne Yoong, the investigator reevaluates the impact of PROGRESA on these different outcomes by accounting for the multiplicity of tests under consideration.
Broader Impact: The projects described above will be useful not only to economists, but to researchers in a wide array of disciplines. Control of the FDR has been suggested in numerous applications, including, among others, educational studies, analysis of microarray data, model selection, and plant breeding. For this reason, the development of more powerful methods for control of the FDR, as described in the first project, will be of use in many applications beyond the evaluation of hedge funds. The estimation of treatment effects lies at the center of not only many economic questions, as evidenced by its large literature within economics, but also many questions in biostatistics. For example, the efficacy of a new drug or therapy may vary with the observable covariates of the patients. It may be of interest to determine for which values of these observed covariates the new drug or therapy is effective, so the results of the second project will therefore be relevant. Finally, randomized trials are, of course, common not only in development economics, but in all parts of the sciences. Hence, the methodology developed in the third project for the evaluation of PROGRESA will be relevant in any such randomized trial provided that there is more than one outcome of interest.
