This project constructs a functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their functional motions in manipulation tasks. The graphical model FOON are developed and learned by observing state change of objects and human manipulations with the objects. A well-trained FOON enables robots to decipher a task goal, seek the correct objects at the desired states, and generate an optimal motion, which eventually allows robots to perform daily living tasks and provide needed help to people with disabilities and seniors. The project builds a kitchen environment with representative kitchen objects and a virtual environment to collect training data and learn a representative FOON from the observations of self-generated manipulation tasks performed in both environments. The data collected in the proposed work enables researchers to fully explore daily living tasks from the object state and object motion point of view and provide excellent training and testing data sets for other related research. The FOON provides a powerful tool for discovering new insights in neuroscience, cognitive science, and psychology. It is also useful in training to break down skills for teaching and training evaluation. This project draws additional minority students through the exceptional diversity programs that exist at USF. The project supports a summer camp to attract young students to STEM careers and the research results are widely disseminated through publications and inclusion in courses.
