The objective of this research is to develop and deploy a cluster testbed hosting multiple terabytes of collective memory. The approach is to develop a distributed kernel-level memory virtualization layer -- a lightweight distributed system that pools together the collective memory of multiple machines across the network.
The intellectual merit of the testbed is in enabling research activities involving unmodified large memory applications in high performance and data intensive computing whose memory requirements cannot be normally accommodated within any single commodity machine. The testbed exploits the key observation that it is an order of magnitude faster to access the memory of another machine over low-latency gigabit Ethernet than it is to access the local disk. Applications executing over the testbed will not require any code modification, recompilation, or relinking. The testbed will feature automated and scalable memory management, distributed resource discovery, jumbo frame support, and fault-tolerance.
The broader impact will be in fulfilling a common compelling need among researchers for an affordable multi-terabyte memory testbed that transparently supports any unmodified application. Once mature, the testbed will be made accessible to other researchers fostering further collaboration. The project will promote diversity by involving a number of minority, women, and undergraduate students in testbed acquisition and development. The testbed will also have a major impact on the high-performance and distributed systems education program at Binghamton University. The PIs will offer courses in 3D imaging and analysis, data mining, security, and virtualization utilizing the testbed platform for course projects.
