Hierarchical models (HM) provide a valuable means of analyzing the kinds of nested data structures that are frequently encountered in social science research (e.g., program participants nested within different evaluation sites). Applications include studies of equity in educational opportunities and attainment, studies of factors central to program success in multi-site evaluation studies, and longitudinal analyses of change in cognitive skills. Whereas most applications of the HM to date have involved the analysis of continuous outcomes, numerous research questions entail the analysis of dichotomous outcomes. While sample sizes in social research settings are frequently modest due to budget constraints, standard approaches to estimation and inference for HMs with dichotomous outcomes become extremely problematic when samples are small to moderate in size. In particular, we run the risk of obtaining misleadingly small confidence intervals for parameters of interest, and our results are potentially highly sensitive to outliers (e.g., a program site at which an intervention was unusually ineffective). A newly developed, general estimation technique termed the Gibbs sampler has been shown to provide a means of obtaining sound parameter estimates and intervals in a variety of complex modeling settings. The purpose of this project is to utilize this technique in developing and implementing estimation strategies that will enable researchers to obtain robust estimates of parameters and appropriate intervals in applications of HMs with dichotomous outcomes in small-sample, social research settings. Guidelines for proper implementation and use of these strategies will be developed through analyses of a series of simulated data sets and through analyses of the data from two studies: A multi-site evaluation of a dropout prevention initiative, and an NSF-funded study of the effects of different mathematics. The software programs that are developed for implementation of these Gibbs-sampling-based estimation strategies will be shared with all interested researchers.
