System for executing encrypted Java programs

An important aspect of protecting software from attack, theft of algorithms, or illegal software use, is eliminating the possibility of performing reverse engineering. One common method to deal with these issues is code obfuscation. However, it is proven to be ineffective. Code encryption is a much more effective means of defying reverse engineering, but it requires managing a secret key available to none but the permissible users. Adequate systems for managing secret keys in a protected trust-zone and supporting execution of encrypted native code have been proposed in the past. Nevertheless, these systems are not suitable as is for protecting managed code. In this paper we propose enhancements to these systems so they support execution of encrypted Java programs that are resistant to reverse engineering. The main difficulty underlying Java protection with encryption is the interpretation that is performed by the JVM. The JVM will require the key to decrypt the encrypted portions of Java code and there is no feasible way of securing the key inside the JVM. To solve this, the authors propose implementing a Java bytecode interpreter inside a trust-zone, governed by a thin hypervisor. This interpreter will run in parallel to the standard JVM, both cooperating to execute encrypted Java programs.