Machine learning studies the design of automatic methods for extracting information from data. It is a highly successful technique that has transformed several fields including computer vision and information retrieval, and it holds great promise to transform many other areas across science and technology. This project aims to substantially advance machine learning by providing new theoretical foundations and algorithms that are required both for new applications and to cope with the current massive amounts of available data. This includes developing well founded techniques for learning more complex objects (than classic techniques that are often limited to prediction), learning from very small amounts of annotated training data, and efficient transfer (of representations and other useful information) among tasks in order to aid more efficient learning of future tasks. These topics are of significant practical importance and expose fundamental statistical and computational issues. This project will impact not only theory of computing and machine learning, but also many application areas where machine learning is used. In addition to advising both graduate and undergraduate students on topics connected to this project, research progress will be integrated in the curricula of several courses at Carnegie Mellow University and course materials will be made available on the web worldwide.
The key research directions of this project are:
(1)Providing formal guarantees and algorithms for transferring internal representations (such as a portion of a deep network) learned while solving an earlier task to new related tasks, in ways that can significantly reduce both data requirements and run-time for solving the new tasks. This project will analyze both a direct transfer of a learned representation, and additional fine-tuning steps using a small amount of data from the new task.
(2)Developing foundations and algorithms for transfer learning in unsupervised and partially supervised learning scenarios, in order to reduce reliance on difficult-to-verify assumptions in cases where labeled data is scarce.
(3)Developing new algorithms for online learning of complex, non-convex functions, which do not satisfy standard conditions such as convexity or Lipschitzness. This project will consider online learning of such functions under various forms of feedback, including full information, bandit, and semi-bandit settings, and will also explore implications of these techniques to transfer learning in partially supervised scenarios.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
