This project develops a parallelizing compiler for distributed shared memory systems, and incorporates a vertically integrated system to manage data and thread locality in parallel programs. The compiler locates threads and data statically whenever sufficient compile time information is available to make near-optimal decisions. When such information is not available, the compiler provides the underlying system with whatever information is available, and lets the underlying system utilize the information. The project investigates the following compile time issues as related to distributed memory systems: (1) automatic parallelism for relaxed models of memory consistency, (2) elimination of false sharing, (3) adaptive coherence protocols, and (4) communication optimizations. Solutions to the issues are implemented within a compiler known as Rochester Optimizing Compiler (ROC). These solutions will be evaluated using standard benchmark programs such as LINPACK, SPEC, Perfect Club, SPLASH and the HPF benchmark suite of Syracuse.