The risk of non response bias is a major threat to the validity of health surveys. Many special studies have found that this bias can impact the results of important health measures. Unfortunately, there is very little guidance about how to evaluate the risk of nonresponse bias. Many surveys rely on the response rate as a key statistic. However, a recent meta-analysis indicates that the response rate is a poor indicator for non response bias. In the studies examined, there was little or no correlation between the response rate and the non response bias. The lack of a good indicator for the risk of bias is harmful to health surveys in two ways. First, surveys with high response rates and relatively high non response bias may be accepted as valid and published. Second, surveys with low response rates and relatively low non response bias may be questioned and rejected for publication. Both of these situations may lead to incorrect conclusions about health policies and practices. The present proposal attempts to fill this void by evaluating a set of indicators for the risk of non response bias. Each of these measures (including the response rate) makes assumptions that are untestable in most practical situations. The goal of this research is to understand the strengths and weaknesses of each of several alternative measures and the implications of incorrect assumptions. These indicators are compared and contrasted through derivation of their key properties. These properties can include a description of how each measure can be used to place bounds on the potential non response bias and the assumptions required to do so. In addition, a simulation study will be conducted to demonstrate how each measure performs under a varied set of conditions. Finally, all the measures will be applied to existing survey data collections. The goal of the proposed project is to aid in the development of a common understanding of a set of measures that can be used to evaluate the risk of non response bias. This should greatly facilitate efforts to evaluate the quality of health-related survey data.
The risk of non response bias is a major threat to the validity of health surveys. Many special studies have found that this bias can impact the results of important health measures. Unfortunately, there is very little guidance regarding how to evaluate the risk of non response bias. The lack of reasonable, commonly-accepted indicators for the risk of bias is harmful to health research in two ways. First, surveys with high response rates and relatively high non response bias may be accepted as valid and published. In this way, biased results may be propagated. Second, surveys with low response rates and relatively low non response bias may be questioned and rejected for publication. Both of these situations may lead to incorrect conclusions about health policies and practices. The present proposal attempts to fill this void by evaluating an alternative set of indicators for the risk of non response bias. This evaluation includes theoretical derivation of their properties, a simulation study, and application to existing survey data collections.
| Nishimura, Raphael; Wagner, James; Elliott, Michael R (2016) Alternative indicators for the risk of non-response bias: a simulation study. Int Stat Rev 84:43-62 |
| Wagner, James (2012) A Comparison of Alternative Indicators for the Risk of Nonresponse Bias. Public Opin Q 76:555-575 |
