Clinical text documents contain a rich set of clinical knowledge that is invaluable for clinical research. Unfortunately, they remain a largely untapped resource since disseminating such data as-is would jeopardize the privacy of patients and reveal protected health information. Computational de-identification is a means to overcome this problem. It involves processing clinical text documents using natural language processing (NLP) tools and techniques, recognizing patient-related individually identifiable information (e.g., names, addresses, and telephone and social security numbers) in the text, and redacting only those identifiers. In this way, patient privacy is protected and clinical knowledge is preserved. Without computational tools, de-identification places a heavy burden on clinicians shoulders, but it is a necessary step for protecting patient privacy as mandated by both the Privacy Rule of the Health Insurance Portability and Accountability Act (HIPAA) and the Privacy Act of 1974. After exploring existing de-identification tools, the U.S. National Library of Medicine (NLM) is developing new software that is capable of de-identifying many kinds of clinical text documents with high accuracy. The software design uses a number of deterministic and probabilistic pattern recognition algorithms and various computational linguistic methods. We are using many large datasets for names, addresses, and organizations, all of which have the potential to identify patients, in order to find and remove such content from the text. The application accepts text documents in plain text or in HL7 format. If documents are provided in an HL7 format, the application makes use of patient related information embedded in various HL7 segments and fields in order to find and remove that information, including typographical errors and misspellings, from the corpus of the text with high accuracy. The application software includes an editor for visualization and markup called the Visual Tagging Tool (VTT). Although designed specifically for tagging identifiers that contain personally identifiable protected health information, VTT will be made publicly available to the greater NLP community for expanded lexical tagging and text annotation. We are beginning a series of studies to assess the success of de-identifying on a large corpus of tagged clinical documents. The preliminary results of this study suggest that computational de-identification methods may attain an accuracy at or better than the level of 99% sensitivity and 99% specificity across a large spectrum of identifiers containing personally identifiable information.
