This research concerns recognition of words in handwritten responses of children in reading comprehension tests. The approach to word recognition will be based on conditional random fields (CRFs), which are discriminative methods that do not make any assumptions about the underlying data and hence are known to be superior to Hidden Markov Modes (HMMs) for sequence labeling problems. The student response is first segmented into word images using an existing neural network based algorithm. Each word image is then over- segmented into a number of small segments such that the combination of segments forms character images. Segments are labeled as characters with probability evaluated from the CRF model. The total probability of a word image representing an entry from the lexicon is computed using a dynamic programming algorithm which evaluates the optimal combination of segments. A lexicon derived from the reading passage, testing prompt, answer rubric and student responses is used to limit the number of paths to explore.
The state and transition parameters of the CRF model are estimated from handwriting samples. State parameters correspond to features such as: position (normalized by length), place (in start, middle or end), height, width, distances to prototype, deviations of height, etc. Transition parameters correspond to features such as: label of character pair (th, er, qu, etc), vertical overlap (of pixels of candidate character images), height difference, width difference, aspect ratio difference, bigram width, etc.
The research test-bed will consist of scored handwritten responses to reading comprehension prompts from Grades 8 and 5 of an inner city school in Buffalo, New York. There are 300 Grade 8 responses and 200 Grade 5 responses, with about 100-150 words in each response. Training data for parameter estimation will initially consist of 150 student responses and 1,000 half-page writings of adults. These will be supplemented with additional school data as research progresses.
Goal-oriented integration of complex document image analysis, natural language processing and machine learning will drive improved handwriting recognition methods. Children?s handwriting recognition has never before been studied in document analysis. Handwriting recognition technology for complex documents is as yet largely unavailable. Success will allow statewide testing to be done later in the school year with results provided sooner thereby having an impact on improved education.
