×

DynOpVm: VM-based software obfuscation with dynamic opcode mapping. (English) Zbl 1458.94225

Deng, Robert H. (ed.) et al., Applied cryptography and network security. 17th international conference, ACNS 2019, Bogota, Colombia, June 5–7, 2019. Proceedings. Cham: Springer. Lect. Notes Comput. Sci. 11464, 155-174 (2019).
Summary: VM-based software obfuscation has emerged as an effective technique for program obfuscation. Despite various attempts in improving its effectiveness and security, existing VM-based software obfuscators use potentially multiple but static secret mappings between virtual and native opcodes to hide the underlying instructions. In this paper, we present an attack using frequency analysis to effectively recover the secret mapping to compromise the protection, and then propose a novel VM-based obfuscator in which each basic block uses a dynamic and control-flow-aware mapping between the virtual and native instructions. We show that our proposed VM-based obfuscator not only renders the frequency analysis attack ineffective, but dictates the execution and program analysis to follow the original control flow of the program, making state-of-the-art backward tainting and slicing ineffective. We implement a prototype of our VM-based obfuscator and show its effectiveness with experiments on SPEC benchmarking and other real-world applications.
For the entire collection see [Zbl 1415.94004].

MSC:

94A60 Cryptography
68P25 Data encryption (aspects in computer science)
PDFBibTeX XMLCite
Full Text: DOI Link