This project is one of a set of special awards to junior investigators, of small amounts of money to allow them to initiative a new research agenda. This project will study the causes of variation in coding of text, by comparing different methods-coding themes or words-across different informants. Fifty informants will write narrative texts of a recent illness. The texts will be analyzed by ten coders using two different methods: marking contiguous segments of text for themes, and performing a content analysis of key words. Coders will do the two tasks in random order. The themes will be tagged in the texts and the structures of themes used by each coder will be analyzed. In addition the 100 key words mentioned most frequently will be coded. A theme by word affiliation matrix will be constructed. All pairs of coders will be analyzed for similarities and differences to identify core themes and words. Prototypical themes and informants will be identified. Tests of intercoder agreement will be conducted to see how many or few are required for a consensus on themes of varying levels of complexity. The research will advance our understanding of the parameters causing variation in coding, a core task in any quantitative analysis of text. It will be a first step towards more extensive research aiming to advance the analysis of texts to a more reliable and scientifically sound methodology.
