An obfuscator is aimed at confusing a program and makes the program source code more difficult to be understood by a hacker and increases the cost of its reverse engineering while preserving the functionality. Generally, software protection is the aim of obfuscation. The main challenges in obfuscation techniques are to obscure their impact on the obfuscated code, resist any automatic deobfuscation technique, and minimize run-time overheads. In addition, due to the semantics gap between programming language semantics and obfuscators, often, obfuscated code could be easily detected, statically. To avoid all the difficulties and shortcomings of the current obfuscation methods, in this thesis a new obfuscation method based on the abstract interpretation is presented. It is possible to define conditions by analyzing and obtaining the interval of variables based upon abstract semantic of the program and they have not to affect the runtime progress of the original program. Then, our proposed obfuscator has embedded some codes that do not have any effect on the runtime progress. So, our proposed obfuscator helps us to reduce the accuracy of abstract interpretation analysis. We aimed to represent an obfuscator to prevent the abstract interpretation of code and reduce the accuracy and soundness of the static analysis. For obfuscation, we implemented an abstract interpretation tool that computes the interval of the flag register on the assembly code. Thus, by adding a jump basic block into the control flow graph of the program based upon the fixed points of a specific flag named c, our obfuscator attempts to increase the complexity of the code. By increasing the number of various executing paths, particularly non-executable loops into the code, tracing the basic blocks in programs source code become much difficult. Jumps and dead codes are added to the program in a way that makes the abstract interpreters unable to detect them. To prevent the abstract interpretation, a basic block with a negative state of flag c the predecessor of jump basic block, NB, is created. We conclude that by considering a sequence of conditional jump basic blocks for which, the false branch of each basic block connects to NBs, we would be able to have potent obfuscators that prevent the abstract interpretation analysis.
Code obfuscation, static analysis, abstract interpretation, Deobfuscation.