Program performance is highly dependent on the amount of memory available to the program. In traditional computing systems, the memory working set of an application has a bounded size - providing more memory to an application improves performance until its working set is met. Once the working set is met, additional memory yields little or no benefit. However, in the presence of garbage collection (a technique for memory management where space that is unlikely to be reused by an application is automatically reclaimed), the relationship between program performance and memory allocation is more complex. Data is managed at three levels: the compiler manages data objects at the program level, the garbage collector manages the heap at the virtual machine level, and the virtual memory manager manages virtual memory at the operating system level. The middle layer plays a critical role. Increasing an application's heap size can reduce the frequency of garbage collections and improve performance, but too large a heap may trigger paging and degrade performance.
Program performance is highly dependent on the amount of memory available to the program. While this is true for all programs, it is especially true for programs using garbage collection. Garbage collection (or, less formally, "dynamic memory management") improves program security and programmer productivity by automating several common, error-prone memory requirements. This improvement comes at a cost of dramatically increasing the program's overall memory demands and, in particular, the amount of memory it uses on a regular basis (its "working set"). When the computer does not have enough free RAM to meet these demands, the performance of garbage collected applications plummets and quickly becomes unusable. While past projects had looked at improving this performance for single programs running on dedicated machines, this project developed techniques that work in the multi-programmed environments that are really used today. This process is complicated by the fact that the relationship between program performance and memory allocation is complex. Data is managed at three levels: the compiler manages data objects at the program level, the garbage collector manages the heap at the virtual machine level, and the virtual memory manager manages virtual memory at the operating system level. Our approach, which we named Poor Richard's Memory Manager, was based off the maxim "a penny saved is a penny earned." It targets that crucial middle level, but does so in a way that is independent of the actual garbage collector. By making our changes small and portable, we can save the performance optimizations that already exist within existing systems. We then used a collaborative approach in which processes work from a common "whiteboard" to make system-wide decisions and insure that they fully utilize memory. The result of these decisions was an algorithm that could be easily added to any garbage collection system (it required under 100 lines of code within the actual GC) and works within any operating system. Our results showed that it improved the performance of every garbage collector on which we tried and could scale from simple single processor systems all the way up to multiprocessor, multicore workstations. This should greatly improve the performance of garbage collection on all these machines and make it easier to gain the program safety and improved productivity these languages provide.
