Analytics (Objective 1) The SSA adjudication process is dynamic, involving a complex sequence of decisions by several offices within SSA as well as the decisions and resources of the claimants themselves. NIH undertook an overarching project, the Adjudication 1 project, to comprehensively model this process. Project goals are to: 1. Develop analytical tools to analyze various aspects of the adjudication process in terms of accuracy, consistency, and timeliness; 2. Develop tools to predict how the system responds to external shocks; 3. Develop methods to analyze data taking into account the multi-stage application process in which data are collected; 4. Develop tools to assist with the disability determination process; 5. Quantify the extent to which SSA can adjust the system to respond to changes; 6. Derive useful statistics to monitor and adjust program performance, based on important outcomes measures (accuracy, timeliness, consistency). Case Flow Processing (Case Status Change Model and Queuing Theory): This project aims to develop methods to analyze system timeliness, measure processing times, and derive optimal flow characteristics. Our work in the area of system timeliness took on two complementary directions. To study system delays, we built a queuing model for the adjudication system that allows the user to obtain system performance statistics. Addressing the separation between wait times and processing time is our second direction of inquiry. For this purpose, we developed a batch processing model. To date, we have scripted code to obtain the needed transition probabilities, determined the rate at which jobs enter the system, completed the queue code, and estimated the distributions of processing/waiting times by using three techniques. Data Mining Feasibility Study:
The aim of this project is to extract functional information from clinical documentation. In order to identify functional information, it is necessary to understand the variation in text used in clinical documentation, noting those rich in functional information and those focusing on health conditions. Therefore, this project has two main goals: characterize medical documentation with respect to functional language; and, extract functional information from samples of medical documentation data. In 2017, the project team focused on functional mobility, as described in the WHOs ICF. The techniques developed from this project will have immediate application to the SSA disability determination process by locating instances where information on claimant function can be found in medical records. Denied CAL The aim of this project is to provide recommendations for changes to SSAs business rules to improve the precision of the CAL software, i.e., increase the allowance rate for cases selected by the CAL software. The NIH team provided SSA with seven recommendations for rule sets for CAL UID 204 (Adult Non-Hodgkins Lymphoma). While SSA reviewed these recommendations, NIH continued to work on the IAA year end project deliverables, specifically, to provide recommendations for all eight CAL conditions initially selected. To that end, we began working with CAL UIDs 62 (Early-Onset Alzheimer Disease), 68 (Idiopathic Pulmonary Fibrosis), and 151 (Malignant Melanoma with Metastases). We generated rule sets for these three conditions using the methods previously developed for CAL UID 29 and 204. After examination of initial results, NIH proceeded to integrate additional pre-processing and feature selection capabilities into existing methods. These steps helped to moderately improve performance while also producing more contained, consistent rule sets. Recommendations for these three CAL conditions (UID 63, 68, 151) were provided to SSA for review in June. The SSA-NIH project team agreed that NIH would continue with analysis of the remaining three CAL conditions while SSA reviews the June outcomes. NIH also began looking at rule simplification methods. These methods are intended to simplify and produce more compact rule sets that can be run as an alternative to SSAs existing business rules rather than an addition to the current rules. While the analytical methods already exist, the relevant software supporting implementation of the methods is rather limited. The objective of this work is to provide SSA the opportunity to select the most appropriate tool for use within the agency. Based on project discussions with SSA, the NIH team also reviewed more general data about the eight selected CAL conditions. From this review, we generated statistics for comparison across conditions. While these statistics might not have a direct impact on the recommended rule sets, the NIH team hopes these data may be informative to SSA and may assist in reviewing the rule sets or CAL cases. CAT development (Objective 2) The NIH/RMD previously awarded a contract to the Boston University Health and Disability Research Institute (BU-HDRI) to develop a comprehensive set of tools to characterize the full continuum of individual capabilities (i.e. human function) relevant to work. This method uses Computer Adaptive Testing (CAT) coupled with Item Response Theory (IRT). The development of CAT tools (also known as the Functional Assessment Battery or FAB) is a sequential process; one step must be completed before advancing to the next step. A calibration study was conducted to refine the item pools for the Daily Activities and Learning and Applying Knowledge. Additionally, Westat, a research survey firm, conducted replenishment testing of items for the first two domains, physical function and behavioral health, as well as legacy validation on all item pools. Boston University has embarked on a number of additional post development studies to enhance the functionality, utility, and comprehensiveness of the instruments. NIH plans to monitor claimant outcomes for a period of two years, assessing the predictive capacity of the instruments developed under this protocol. Upon expiration of the period of performance for this contract, BU-HDRI submitted a final report detailing efforts during the collaboration. NIH executed a new contract with BU this FY to continue refinement of the instruments and explore implementation opportunities within the SSA process. We completed a national study to replenish the items utilized in the IRT/CAT instrument.
Showing the most recent 10 out of 12 publications
