With the increasing use of artificial intelligence (AI) systems in life-changing decisions, such as hiring or firing of individuals or the length of jail sentences, there has been an increasing concern about the fairness of these systems. There is a need to guarantee that AI systems are not biased against segments of the population. This project aims to mitigate AI bias in the domain of computer vision, a driving application for much of the recent advances in a popular form of AI known as deep learning. Computer vision systems are increasingly prevalent in areas of society ranging from healthcare to law enforcement: from apps that analyze skin pictures for melanoma detection to face recognition systems used in criminal investigations. These systems are subject to three major sources of bias: biased data, biased annotations, and biased models. Biased data follows from poor image collection practices, typically the under-representation of certain population groups. Biased annotation follows from the use of annotation platforms with untrained image labelers, who tend to produce annotations that reflect their own image interpretations, rather than objective labels. Biased models can ensue from either the existence of data or annotation biases on the datasets used to train the models, or the choice of biased model architectures. The three bias components have received different attention in the literature, with most previous work focusing on the mitigation of model bias. However, this usually boils down to downplaying groups for which there is a lot of data and promoting groups for which data is scarce. This practice can hurt overall system performance. The remaining sources of bias, datasets and annotation, have received very little algorithmic attention.
The project aims to overcome this problem, by introducing a new framework to jointly address the three sources of bias within one unified bias mitigation architecture. This architecture aims to train fair classifiers by iterative optimization of three distinct modules: 1) Dataset bias mitigation algorithms that identify and downweigh biased examples and seek additional examples in a large pool of data to counterbalance the associated biases. 2) Label bias mitigation systems based on machine teaching algorithms that establish clear, replicable, and auditable procedures to teach annotators how to label images without label bias. 3) Model auditing techniques based on counterfactual visual explanations that enable the visualization of the factors contributing to model decisions and why they are biased. The three modules combine into an architecture for joint dataset, label, and model bias mitigation by iterative optimization of datasets, annotators, and models to minimize bias. The project will generate software for dataset bias mitigation, unbiased annotator training, explanations and visualizations, model auditing, and fair model training, which will be made available from the investigator website. This will be complemented with datasets for the design of various form of bias mitigation algorithms, and tools to help practitioners detect and combat bias. Several activities are also planned to broaden the participation of underrepresented K-12 and undergraduate students in the STEM field. They will include the participation of a team of such students, recruited from University of California San Diego programs that aim to increase the participation of these groups in STEM, and aim to provide these students with early exposure to the challenges of real-world engineering, fair machine learning, and deep learning systems.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
