Analytics (Objective 1) Introduction and Overview We have aligned our research with the current SSA strategic plan. Thus, NIH research supports SSA objectives in four focus areas: 1) Case Prioritization and Eligibility, 2) Adjudicator Support Tools, 3) Quality and Productivity Tools, and 4) Functional Assessment Tools. In each of these areas, several subprojects have been initiated, are in progress, or have been completed, each with its own aims as articulated below. Focus areas 1-3: Objectives, summary of findings, and major accomplishments 1) Case Prioritization and Eligibility The objective of this focus area is to automate the collection and analysis of applicant data to improve the efficiency of SSA processes that rank cases based upon adjudication complexity, types of medical allegations, and other criteria of interest. Compassionate Allowance (CAL) case identification: One way that the SSA prioritizes its cases is to expedite decisions for those applicants who have a diagnosis that suggests that they have a high likelihood of being allowed. SSA launched the Compassionate Allowances (CAL) program in 2008, which was the latest of several expedited awards initiatives intended to fast-track applicants who clearly met SSAs definition of disability. Currently, SSA uses software to automatically select cases for the CAL program.
The aim of this subproject is to improve the precision of this software, using various data science methods. Recommendations are intended to be simple, interpretable, able to reduce false positives while minimizing increases in false negatives, and compatible with SSAs domain knowledge, while still being data-driven. To date, we have provided SSA with recommendations for ten CAL conditions, and we will provide an additional three CAL conditions by the end of FY2018. 2) Adjudicator support tools Initial SSA decisions are made by examiners who have very little time to review evidence for a case, usually in the form of lengthy medical records from various healthcare providers. The objective of this focus area is to develop, adapt, and apply methods that aid SSA disability adjudicators to reach accurate, consistent, and timely decisions in accordance with SSA regulations and available medical evidence. Identification of functional terminology:
The aim of this subproject is to extract functional information from clinical documentation, which is an underdeveloped area of machine learning and NLP. While functional information plays an important role in SSAs disability determination process, it is not readily available. It requires considerable time, effort, and expense for SSA adjudicators to collect and summarize claimants functional information from medical records since the process is performed manually. Automating the extraction, retrieval, and classification of functional information will improve the efficiency of SSAs processes. 3) Quality and productivity tools The objective of this focus area is to provide a suite of tools for managers to understand and, if necessary, adjust SSA employee workloads. These methods also provide diagnostic and analysis tools for managers to track workflow on a system level. Case flow processing:
The aim of this subproject is to develop methods to analyze system timeliness, measure processing times, and derive optimal flow characteristics of cases through various stages of the SSA disability adjudication process. We deployed a deterministic queueing network based on a coarse picture of workflow at the office-level. From this model, we identified transitions into and out of certain status codes, which was sufficient for understanding bottlenecks and backlog. Using this network, SSA can identify the location of these bottlenecks in each office per month by determining which of the holding status codes are applied to more cases. We also provided a tool for SSAs use to compute the flow of cases in the queueing network. In this tool, one can modify characteristics of the queueing network on the office-level and determine how case flow would change as a result. We met with SSA periodically in 2017 and 2018 to provide training and transfer our analytic code, user guide, and methods for ongoing use and refinement by SSA. The objectives for this project were accomplished and it was closed at the end of FY2018. Data envelopment method:
The aim of this subproject is to assess the efficiency of individual SSA hearing offices that adjudicate appeals. We developed a multi-stage data envelopment analysis method to estimate the performance of 167 hearing offices. These methods provide the agency with key information about the levels of performance across offices. In particular, they inform the identification of more efficient hearing offices and establish best practice for enhancing the efficiency of others. The developed model is a powerful assessment tool that, in combination with additional data, could inform policy recommendations. We met with SSA periodically in 2017 and 2018 to transfer our analytic code, user guide, and methods for ongoing use and refinement by SSA. The objectives for this project were accomplished and it was closed at the end of FY2018. WD-FAB development (Objective 2) Introduction and Overview Development and testing of the Work Disability Functional Assessment Battery (WD-FAB) relates to the fourth focus area in support of SSAs current operational objectives, i.e. Functional Assessment Tools. This year, development of the WD-FAB has focused on completion of the instrument and score interpretation. In addition, SSA has commissioned a study design report for a large scale, applied study of the WD-FAB to consider for potential implementation. Background In collaboration with the SSA, the NIH and Boston University developed a comprehensive and efficient work disability functional assessment battery. Contemporary models of disability indicate that in order to assess work disability, what individuals can do and what they are expected to do for work must both be assessed. The WD-FAB is intended to assess what individuals can do. The WD-FAB is a 15-20-minute individualized assessment of functional activity that uses Item Response Theory (IRT), along with computer adaptive technology (CAT) to select the most relevant test items from a large pool of items to measure self-reported functional ability. Item-based scoring means respondents do not need to answer all items or the same items to obtain comparative scores and scores are obtained in a highly efficient manner. Focus area 4: Objectives, summary of findings and major accomplishments 4) Functional Assessment Tools The objective of this focus area is to develop new ways to collect, structure, and interpret functional data for use by SSA. This work will include development of the WD-FAB and methods to assist in interpreting WD-FAB results. WD-FAB instrument development:
The aim of this subproject is to finalize the development of the WD-FAB so that it is ready for real-world, applied testing. The instrument now includes over 300 items across eight domains, four of which represent physical function (basic mobility, upper body function, fine motor function, community mobility) and four of which represent mental health function (communication & cognition, resilience & sociability, self-regulation, and mood & emotions). Functional stages (e.g., low, moderate, high functioning) were developed by content experts to aid score interpretation. To date, the reliability and validity of the WD-FAB have been supported by a variety of evidence from a continuum of studies. Continuing Disability Review study design: Once an individual is awarded disability benefits, their disability status is reassessed periodically. Following development of the WD-FAB, SSA requested creation of a study
Showing the most recent 10 out of 12 publications
