Advancements in technology over the past decade are leading to a promising future for pervasive distributed computing (PDC), where access to information and services is provided anytime and anywhere around the world. The proliferation of Internet-scale applications poses new challenges for how future file systems are designed and managed to enable this future. The difficulty of these challenges grows with the number of users and the intensity of the data, and is further compounded by the need to support replication and consistency. However, previous file replication and consistency maintenance approaches are not efficient enough to be scalable for PDC. These two issues are typically addressed separately, despite the significant interdependencies between them. This research addresses this issue through the development of a file management system with Efficient and Reciprocal file Replication and Consistency maintenance (ERRC). The ERRC system will incorporate swarm intelligence based file replication and coordinated multi-factor-oriented file replication algorithms to achieve high efficiency in both file replication and consistency maintenance. The system also contains self-adaptive file replication and consistency maintenance mechanism that allows each node to determine the need of file replication and consistency maintenance in a decentralized manner. Understanding and insight gained as a result of this project will be disseminated through technology transfer to industry partners, in addition to publication and software release channels. The multi-disciplinary nature of this research also lends itself to cross-disciplinary education and well-rounded training of students.
The proliferation of the Internet-scale applications poses new challenges of how future file systems are designed and managed to enable pervasive information and service access. The difficulty of these challenges grows with the number of users and the intensity of the data. This is further compounded by the need of support for replication and consistency. Thus, the issues of File Replication (FR) and Consistency Maintenance (CM) must be addressed in order to ensure the successful deployment of PDC. However, previous FR and CM approaches are not sufficiently scalable and efficient for PDC. More importantly, these two issues are typically addressed separately, despite the significant interdependencies between them which are vital for scalability and efficiency: FR should proactively reduce unnecessary replicas to minimize the overhead of CM, which in turn provides guarantee for the fidelity of consistency among replicas. Intellectual Merits: A file management system enabling the reciprocity between FR and CM is vital for the promising vision. This project is devoted to the research of addressing this need through the development of a file management system with Efficient and Reciprocal FR and CM (ERRC). We have investigated, developed and implemented ERRC on top of a peer-to-peer overlay by leveraging its data storage and overlay routing. The ERRC system is built on four key innovations. (1) Swarm intelligence based FR. (2) Coordinated multi-factor-oriented FR. (3) Self-adaptive FR and CM. (4) Proximity-aware poll-based and push-based CM. Broader Impacts: The broader impacts of this project include education and outreach. This project has been used for course project and produces collaborative research opportunities for graduate and undergraduate students and faculty. Several REU students were involved in this project. Further, this project provides critical insight into FR and CM, which has significant impact on a variety of commonly used PDC applications. The research results have been disseminated through publication. This research serves the PDC community as a vehicle to conduct further research and experiments.
