Reverse Engineering Self-Modifying Code: Unpacker Extraction

An important application of binary-level reverse engineering is in reconstructing the internal logic of computer malware. Most malware code is distributed in encrypted (or "packed") form, at runtime, an unpacker routine transforms this to the original executable form of the code, which is...

Full description

Saved in:
Bibliographic Details
Main Authors: Debray, S, Patel, J
Format: Conference Proceeding
Language:English
Subjects:
binary analysis
Malware
malware analysis
Payloads
Registers
Reverse engineering
Runtime
self-modifying code
Semantics
Weapons
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An important application of binary-level reverse engineering is in reconstructing the internal logic of computer malware. Most malware code is distributed in encrypted (or "packed") form, at runtime, an unpacker routine transforms this to the original executable form of the code, which is then executed. Most of the existing work on analysis of such programs focuses on detecting unpacking and extracting the unpacked code. However, this does not shed any light on the functionality of different portions of the code so obtained, and in particular does not distinguish between code that performs unpacking and code that does not, identifying such functionality can be helpful for reverse engineering the code. This paper describes a technique for identifying and extracting the unpacker code in a self-modifying program. Our algorithm uses offline analysis of a dynamic instruction trace both to identify the point(s) where unpacking occurs and to identify and extract the corresponding unpacker code.
ISSN:1095-1350
2375-5369
DOI:10.1109/WCRE.2010.22