Mobile computer systems and software are increasingly subject to a host of security threats and malicious software (malware) attacks due to vulnerabilities in their coding. The difficulty in achieving secure systems is further compounded by prevalence of unsophisticated users and the increasing reliance on third party software integration by enabling software module plugins for such user applications as web browsers and search engines. Traditional approaches have sought to provide an absolute defense to specific malware attacks by patching software vulnerabilities or detecting and blocking malware. One difficulty with these approaches for small mobile platforms is that the design constraints on these devices often favor low power to maximize battery life over enhancements to support security protocols. The current situation also represents a programmatic arms race between patching existing vulnerabilities and exploiting vulnerabilities in new application code. This research develops a new secure mobile computing environment based on current mobile technology widely available as consumer end products that seeks to use program differentiation to reduce the propagation rate of malware when a software vulnerability exists. This results not in the direct elimination of security vulnerabilities, but in the dramatic reduction in scope of any security exploit to infect large numbers of systems. By constraining the outbreak to only a few systems, counter measures can be employed before significant economic damage can result. By modifying aspects of the execution of the application, application executables can be permuted into unique versions for each distributed instance. Differentiation is achieved using hardware and/or systems software modifications. Areas of differentiation include function call/return and system call semantics, as well as a proposal for hardware-supported Instruction Register File access and intrusion detection monitoring. Differentiation of executables hinders analysis for vulnerabilities as well as prevents the exploitation of a vulnerability in a single distributed version from propagating to other instances of that application. By focusing on prevention of malware propagation in addition to traditional absolute defenses, we target the economics of malware in order to make attacks prohibitively expensive and infeasible. This approach can be more feasible than some conventional security approaches since it can be readily applied to the restricted performance capabilities of mobile systems and the legal constraints on the export of encryption technology. Further information and software distribution related to this project can be accessed at: www.cs.fsu.edu/~tyson/differentiation.
We used the time in the extension year of the grant to study the efficacy of using the permutation techniques to obfuscate application code for Android devices. Three techniques to limit the impact of malware on mobile systems were studied: (1) a simple encryption scheme in which the distribution code is transmitted in an encrypted form and dynamic compilation performs decryption; (2) manipulating control flow locations within the application using indirection and permutation to obfuscate execution behavior; (3) augmenting application code to include analysis code which determines whether the application code has been modified by some advisory. Each of these approaches seeks to prevent application code from being manipulated by malicious developers. Each has weaknesses and all three approaches can be used together to improve the ability of an application developer to protect code from unauthorized modification. Encryption by itself is only marginally successful in preventing third party modification of application code. The best of these systems distributes application code using public key encryption, enabling users to decrypt execution code within the "just-in-time" compiler. Unfortunately, this does not prevent reverse engineering the code since the entire runtime system is available as open source, enabling anyone to see the decrypted version of the application code. It does prevent an adversary from modifying, then repackaging the code using the same encryption key; however, many mobile application downloads are performed by users unconcerned about whether an application is repackaged and distributed using another key or in an un-encrypted form. While encryption cannot prevent repackaging of modified application by a third party, it does eliminate the ability of that third party to encrypt the repackage application using the original encryption key, thereby preventing users from unknowingly downloading a compromised application. The primary method developed in this research was to examine the application of indirection in specifying control flow and the permutation of the jump tables used to perform that indirection. These techniques were used to obfuscate individual instances of an application. Initial evaluation was performed on the executable image. However, this approach must be modified in mobile applications since the distribution image of an application is a higher-level virtual machine code. Instead of permuting the return address stack, we instead permute the links into the virtual machine and API libraries resident within the virtual machine (as opposed to any libraries distributed with the application. While this does not remove the ability to repackage applications, it does provide control over how system resources can be accessed on each machine. The third technique is one we developed specifically to prevent reverse engineering and repackaging of modified mobile software. The basic idea is to incorporate additional computation into the application to identify when the program has been modified. This could be as simple as incorporating a checksum or CRC in the Dalvik executable file. To discourage reverse engineering, it is necessary obfuscate the check placed in the application code. This is achieved by interleaving the additional computation in with the application code. Function call graph analysis and dataflow analysis techniques enable automated code insertion by the compiler to protect a wide portion of the application code. The application developer also has a variety of options on how to respond to code deemed to have been altered, from aborting the execution to changing the application behavior. We have implemented this technique on an application we have developed to determine required overhead in software development and impact on execution, and we believe that this approach is viable to discourage repackaging of popular applications.
