Humans often reason about their observations abstractly to prevent themselves from drawing incorrect conclusions based on unimportant differences. For example, a person tries to avoid on-coming traffic regardless of the lighting conditions; in this case, illumination is said to be an invariance to the problem of traffic avoidance. The goal of this project is to create methods and algorithms to make decisions by learning representations of data with invariances. Beyond simple invariances such as "rotating an image does not change whether it shows a cat," this project seeks to learn more flexible forms of invariances. Some examples include: more general transformations such as changes in pose and facial expressions, semantic or logic relationships between classes (e.g., an image cannot be determined as both having and not having a cat), and structured relationships between entities (e.g., adding or removing an edge in a user's social network does not change that user's preferences). The result will benefit a wide range of social and real-world applications including computer vision, natural language processing, and graph-structured data analysis.
This project will use tools from functional analysis and optimization theory to achieve these goals while retaining the scalability, modularity, reliability, and flexibility of existing methods without these invariances. Specifically, it will apply linear and sublinear regularizations on a reproducing kernel Hilbert space to introduce invariant representations in the resulting Hilbert or Banach spaces. Three thrusts will be pursued. First, generalized invariances will be incorporated into distance and similarity measures between multiple domains, allowing transferrable feature representations to be inferred across domains. The result will be used for transfer learning such as few-shot prediction and multi-way relationship modeling. Second, logical relationships between classes will be modeled by kernels on labels, which, when applied in conjunction with adversarial training, can significantly improve learning under a shifting distribution of input and output. Third, invariances will be built into convex neural networks, allowing invariant features to be learned across tasks through the intermediate layers. The data and algorithm implementations resulting from the project will be disseminated publicly, under permissive open-source licenses.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
