Behavioral, social and biomedical researchers use a variety of instruments to measure a wide range of personal, environmental, behavioral and biomedical variables. Before a new measurement scale is adopted for practical use, one needs to evaluate inter, intra and total agreement among various observers and/or measurement methods. In this proposal we take a fresh look into the field of assessing observer agreement for continuous measurement scales. Existing indices of agreement, such as the intra-class correlation coefficient, indices based on generalizability theory (GT) and the concordance correlation coefficient measure the total observer agreement, which combines the inter and intra observer components of agreement. Currently there is no proper inter-observer agreement index for assessing the """"""""true"""""""" agreement among observers in terms of true (i.e., error-free) values of different observers on the same subject. The first major objective of this project is to develop two new indices of interobserver agreement defined in terms of the distances among the true values assigned to each subject by several observers. In the proposed research, we will develop methods for statistical inference on the new interobserver agreement indices and investigate their properties via simulations. ? The second major objective of this project is to develop and investigate innovative methods for statistical inference on the GT-based indices. In GT it is usually assumed that the observations follow an analysis of variance (ANOVA) model and inference is made under the normality assumption. These assumptions are often unrealistic. We will extend GT concepts to more general settings and develop nonparametric, Semi-parametric and parametric methods for estimation and inference. Existing and new agreement indices will be compared and linked. All the proposed methods will be applied to real data sets from the behavioral and biomedical sciences. ? ?
| Haber, Michael; Gao, Jingjing; Barnhart, Huiman X (2010) Evaluation of Agreement between Measurement Methods from Data with Matched Repeated Measurements via the Coefficient of Individual Agreement. J Data Sci 8:457-469 |
| Pan, Yi; Gao, Jingjing; Haber, Michael et al. (2010) Estimation of coefficients of individual agreement (CIAs) for quantitative and binary data using SAS and R. Comput Methods Programs Biomed 98:214-9 |
| Haber, Michael; Barnhart, Huiman X (2008) A general approach to evaluating agreement between two observers or methods of measurement from quantitative data with replicated measurements. Stat Methods Med Res 17:151-69 |
| Barnhart, Huiman X; Kosinski, Andrzej S; Haber, Michael J (2007) Assessing individual agreement. J Biopharm Stat 17:697-719 |
| Barnhart, Huiman X; Haber, Michael J; Lin, Lawrence I (2007) An overview on assessing agreement with continuous measurements. J Biopharm Stat 17:529-69 |
| Crawford, Sara B; Kosinski, Andrzej S; Lin, Hung-Mo et al. (2007) Computer programs for the concordance correlation coefficient. Comput Methods Programs Biomed 88:62-74 |
| Haber, Michael; Gao, Jingjing; Barnhart, Huiman X (2007) Assessing observer agreement in studies involving replicated binary observations. J Biopharm Stat 17:757-66 |
| Barnhart, Huiman X; Lokhnygina, Yuliya; Kosinski, Andrzej S et al. (2007) Comparison of concordance correlation coefficient and coefficient of individual agreement in assessing agreement. J Biopharm Stat 17:721-38 |
| Haber, Michael; Barnhart, Huiman X (2006) Coefficients of agreement for fixed observers. Stat Methods Med Res 15:255-71 |
| Barnhart, Huiman X; Song, Jingli; Haber, Michael J (2005) Assessing intra, inter and total agreement with replicated readings. Stat Med 24:1371-84 |
