The past decade has witnessed tremendous advances in computing, wired and wireless communication and storage technologies. It is also important that remarkable cost reductions have made large computing and storage capacity increasingly available. Many networks are integrated into the world's largest IT (Information Technology) infrastructure called the Internet. With the unprecedented connectivity provided by Internet, a huge amount digital data has been created and made available. Storage for the huge amount of data being generated has become a significant challenge. At the same time, more data types like audio and video are converted into digital format and data in various types needs to be preserved for a very long term (beyond 100 years). Thus, we are indeed in a big data era.
The big data problem is currently faced by not only individuals, but also by almost all segments of the society, including professional groups, research communities, companies and government. In this project to address the challenges of storing and accessing big data, the plan is to investigate new and efficient ways of using a promising technology called Shingled Magnetic Recording (SMR) for storing large volumes of data in magnetic disk drives (called Shingled Write Disks). SMR based drives are a promising technology that may answer the storage demand of big data. Since the benefits of SMR come from overlapping tracks during writes, it is important to investigate novel uses to reduce the potential write amplification when updates of data are performed. The project plan is to develop both novel layout designs that can reduce write amplification and specialized file systems that can enable applications to use such a device without any modifications, as well as, novel ways of efficiently supporting tree and graph based data structures on SMR drives for many big data related applications. The successful completion of this work will provide a new type of storage devices meeting the demand of big data.
