Unpacker: A Practical Guide to Modular Malware Packer Detection and Unpacking
quality 7/10 · good
0 net
AI Summary
Unpacker is a modular malware packer detection and unpacking tool that automatically identifies packers (UPX, ASPack, Themida, VMProtect, MPRESS) via signatures, entropy, and heuristics, then dispatches to the appropriate unpacker module—native decompression for UPX, emulation-based unpacking via Unicorn/Qiling for others—with built-in validation using string analysis and file metadata.
Tags
Entities
Unpacker
Andrey Pautov
InfoSec Write-ups
UPX
ASPack
Themida
VMProtect
MPRESS
Unicorn
Unipacker
Qiling
String Analyzer
fileinfo.py
Unpacker: A Practical Guide to Modular Malware Packer Detection and Unpacking | by Andrey Pautov | in InfoSec Write-ups - Freedium
Milestone: 20GB Reached
We’ve reached 20GB of stored data — thank you for helping us grow!
Patreon
Ko-fi
Liberapay
Close
< Go to the original
Unpacker: A Practical Guide to Modular Malware Packer Detection and Unpacking
Extract and validate unpacked PE/ELF samples with real examples — and prove it using String Analyzer and File Metadata tools.
Andrey Pautov
Follow
InfoSec Write-ups
·
~14 min read
·
February 28, 2026 (Updated: March 13, 2026)
·
Free: Yes
GitHub - anpa1200/Unpacker
Contribute to anpa1200/Unpacker development by creating an account on GitHub.
github.com
Table of Contents
Introduction
Git repository
What is Unpacker?
Architecture at a Glance
Unpacking techniques used in this tool
Detection: How the Tool Knows What's Packed
Unpackers: One Module per Packer
Running the Pipeline
Validating Unpacking: Why Proof Matters
Validation with String Analyzer
Validation with Basic File Information Gathering Script
End-to-End Workflow
Limitations and Tips
Summary and References
Introduction
Packed malware hides real code behind compression or encryption. To analyze behavior, extract indicators, or compare with threat intel, you first need to unpack the sample. Doing it by hand is tedious and error-prone; using a single rigid tool often fails when the packer isn't the one it was built for.
Unpacker is a modular pipeline: it detects the packer (UPX, ASPack, Themida, VMProtect, etc.), dispatches to the right unpacker module, and outputs a dump you can then analyze. This guide walks through each unpacker with real samples and proofs , and shows how to check and validate correct unpacking using two companion tools you may already use:
String Analyzer — extract and categorize strings, entropy, and obfuscation hints.
Basic File Information Gathering Script (fileinfo.py) — hashes, PE/ELF metadata, packing heuristics, and deep static analysis.
Both tools are read-only (no execution, no decompilation), so you can safely validate unpacked files in automation or air-gapped labs.
Git repository
The project is developed in the open and available on GitHub:
Repository: https://github.com/anpa1200/Unpacker
Clone and install: git clone https://github.com/anpa1200/Unpacker.git cd Unpacker pip install -e .
Run without install (from project root):
python scripts/run_unpacker.py /path/to/sample.exe -o ./unpacked
The repo contains the full source (orchestrator, detector, unpackers), config, scripts, tests, and the in-repo version of this guide. Samples and unpacked results are not committed; use your own samples or the download script (see README) to fetch samples by packer type.
What is Unpacker?
Unpacker is a modular malware packer detection and unpacking tool. It provides:
Orchestrator — Runs: detect → select unpacker → unpack → (optional) PE rebuilder; supports multi-layer unpacking (e.g. VMProtect over UPX).
Multi-method detector — Signatures, section names, entropy, and heuristics; outputs a packer id and confidence ; supports PE and ELF .
Pluggable unpackers — One module per packer (UPX, ASPack, MPRESS, Themida, VMProtect) plus a generic fallback; same interface for all.
You run one command; the tool picks the right unpacker and writes the result to an output directory. You then validate the result with String Analyzer and fileinfo to prove the sample is really unpacked (e.g. lower entropy, larger size, more readable APIs/strings).
Architecture at a Glance
sample.exe
→ Detector (signatures, sections, entropy, heuristics)
→ packer_id (e.g. "aspack") + confidence
→ Dispatcher selects Unpacker (e.g. ASPackUnpacker)
→ Unpack (native UPX, or Unipacker emulation for ASPack/Themida/VMProtect)
→ output: sample.unpacked.aspack.exe
→ (optional) multi-layer: detect again and repeat
Config: config/config.yaml # Unpacker configuration
detector:
confidence_threshold: 0.5
use_signatures: true
use_sections: true
use_entropy: true
use_heuristics: true
# signatures_dir: data/signatures
orchestrator:
max_layers: 5
timeout_seconds: 300
rebuild_iat: true
unpackers:
# Prefer native UPX when available
upx_prefer_native: true
Detection: src/unpacker/detector/ (pipeline, sections, entropy, heuristics, signatures)
Unpackers: src/unpacker/unpackers/ (UPX, ASPack, MPRESS, Themida, VMProtect, generic)
CLI: scripts/run_unpacker.py or unpacker after pip install -e .
Unpacking techniques used in this tool
The tool uses different unpacking techniques depending on the packer and binary format. Below is a more detailed explanation of each.
Detection (before unpacking): Section names, file-content fallbacks (e.g. "ASPack", "VMProtect" in the binary), path hints (path contains vmprotect / themida ), entropy, and heuristics. Best match by confidence selects the unpacker.
Unpacking by technique:
Native decompression (UPX) — System upx -d is called. The format is known; UPX decodes and writes the decompressed image. No emulation. Works for PE and ELF.
Emulation + dump — Unipacker (PE32 only) — Used for ASPack, Themida, and VMProtect 32-bit. The PE is loaded in the Unicorn emulator via Unipacker . Execution starts at the entry point ; the engine runs until it detects an "unpacking done" condition (section hop, write+execute, or packer-specific logic). Then it dumps the in-memory image to a new PE file. For ASPack, Unipacker uses built-in logic; for Themida/VMProtect it uses a generic "unknown" strategy (emulate until heuristic, then dump). This repo patches Unipacker for robustness: safe page-by-page memory read (avoids crashes on unmapped regions) and robust dump (if import fix fails, zero IAT and still write the dump).
Emulation + dump — Qiling (PE32+ / 64-bit VMProtect) — When the sample is 64-bit, Unipacker cannot run. The tool uses Qiling with a Windows rootfs: load the PE, run emulation with a timeout , then read the loaded image from memory (base + SizeOfImage) and write it to disk. Generic "run then dump"; heavy protectors may only partially unpack.
Stub (MPRESS, generic) — Detection may identify the packer, but the unpacker is not implemented; the pipeline returns an error.
Summary: UPX is direct decompression; ASPack/Themida/VMProtect use emulation-based unpacking (run packed code in an emulator until the real code is in memory, then dump). Multi-layer unpacking re-runs detection on the output and repeats up to max_layers .
More detail:
UPX: The format is public; UPX stores compressed data in known sections. The tool calls upx -d ; no execution of the packed binary. UPX decompresses and writes the restored PE/ELF.
Unipacker (PE32): The packer stub at the entry point allocates memory, decompresses the real code, then jumps to it (OEP). We run the stub in Unicorn; Windows APIs are stubbed. We dump when (1) execution section hops to a runtime-written section, (2) a write+execute region is entered, or (3) packer-specific logic (ASPack) says so. Dump = read memory from image base, build PE, fix IAT. Our patches: safe page-by-page read (unmapped → zeros) and robust dump (if IAT fix fails, zero import dir and still write). Unipacker is 32-bit only.
Qiling (PE32+ VMProtect): Full system emulator with Windows rootfs. We run with a timeout and dump the image from memory; no "unpack done" heuristic, so the dump may be partial. IAT is not fixed in this path.
Stub: No implementation; clear error returned.
Is it safe to run real (packed) code in the emulator? The packed code runs inside the emulator , not natively on your CPU. Instructions are interpreted by Unicorn (Unipacker) or Qiling; when the code "calls" Windows APIs, stubs run instead of the real OS — they typically only update emulated state (e.g. allocate emulated memory, return a fake handle). So the sample cannot directly access your real filesystem, network, or hardware unless a stub explicitly does something on the host. Unipacker's stubs are designed to simulate behavior, not to perform real dangerous operations. Qiling can map host paths if configured; use a self-contained rootfs and do not map sensitive host directories. Recommendation: treat all samples as hostile and run the unpacker in a VM or dedicated analysis machine ; do not run on production or personal systems. UPX unpacking does not run the sample at all (only decompression), so it is safe from a "running code" perspective.
Detection: How the Tool Knows What's Packed
The detector merges several methods and returns the best packer_id (and confidence) for the sample.
Section names — Characteristic section names map to packers (case-insensitive), e.g.:
UPX0 / UPX1 → upx (0.98)
.MPRESS1 / .MPRESS2 → mpress (0.98)
.vmp0 / .vmp1 / .vmp2 → vmprotect (0.95)
ASPack , .aspack , .adata → aspack (0.90)
Themida → themida (0.90)
Fallbacks — If section names don't match, the pipeline can use file-content hints (e.g. presence of "ASPack", "VMProtect", ".vmp0" in the binary) or path hints (e.g. path contains vmprotect or themida ) so samples from known packer folders still get the right unpacker.
Entropy — High section entropy suggests packed/compressed content; can yield unknown with a confidence score.
Heuristics — Entry point in last section, few imports, etc., often reported as unknown when no section/signature match.
So when you run the pipeline, you'll see output like:
Detected: aspack (confidence=0.9, method=sections) or Detected: vmprotect (confidence=0.72, method=path_hint) .
Unpackers: One Module per Packer
Each unpacker implements the same interface: it takes a sample path and options (output dir, timeout) and returns success/failure and an output path (or buffer).
UPX
Method: Prefers system upx -d (native decompression). No Python fallback in the default build.
Detection: Section names UPX0 / UPX1 , or (for ELF) scan for UPX0 / UPX1 in the file if section headers are stripped.
Samples: Works on PE and ELF. Place UPX-packed samples in e.g. samples_by_packer/upx/ .
Output: sample.unpacked.upx.exe (or .bin for ELF).
Validation: Unpacked file is larger, entropy drops, and the detector no longer reports UPX (or reports "not packed"). Use String Analyzer to see more APIs/readable strings; use fileinfo to compare size and entropy.
ASPack
Method: Unipacker (emulation-based). Unipacker knows ASPack; it emulates from the entry point and dumps when the real code is unpacked. The project applies patches for safe memory read and robust dump (e.g. surviving fix_imports failures).
Detection: Section names ASPack / .aspack / .adata , or file-content fallback (presence of "ASPack"/"aspack"/".aspack" in the binary).
Samples: e.g. NotePad_aspack.exe in samples_by_packer/aspack/ . Real proof below uses this sample.
Output: sample.unpacked.aspack.exe .
Validation: Same idea: larger file , lower entropy , more readable strings . We'll show concrete String Analyzer and fileinfo output next.
MPRESS
Method: Stub in the open-source repo (no real unpacking yet). Detector can identify MPRESS by section names .MPRESS1 / .MPRESS2 .
Use: Detection works; unpacking returns an error until the module is implemented.
Themida
Method: Unipacker in "unknown" packer mode: emulate from the entry point until section hopping or write+execute is detected, then dump. Themida is not in Unipacker's built-in list, so it's treated as generic/unknown.
Detection: Section name "Themida", or path hint (path contains themida ).
Samples: e.g. in samples_by_packer/themida/ . PE32 only (Unipacker); PE32+ fails with "Not a valid PE file". Complex samples may time out ; increase --timeout if needed.
Output: sample.unpacked.themida.exe .
Validation: Same as others: compare entropy, size, and string/API visibility with String Analyzer and fileinfo.
VMProtect
Method: Same as Themida: Unipacker in unknown mode (emulation then dump).
Detection: Section names .vmp0 / .vmp1 / .vmp2 , or file-content/VMProtect path hint.
Samples: e.g. in samples_by_packer/vmprotect/ . PE32 only ; 64-bit (PE32+) fails with "Not a valid PE file". On 32-bit samples the pipeline can apply multiple layers (VMProtect unpacked several times until max layers).
Output: sample.unpacked.vmprotect.exe (and further layers if still detected as packed).
Validation: Again, entropy drop, size growth, and richer strings/metadata prove successful unpacking.
Generic
Method: Stub — returns "Generic unpacker stub". Used when the detector reports unknown (or generic) and no specific unpacker matches.
Use: Placeholder for future implementation (e.g. Qiling, Speakeasy, or PE-sieve–style dumping for unknown/custom packers).
Running the Pipeline
From the Unpacker project root: # Basic run: detect + unpack, output to ./unpacked
python scripts/run_unpacker.py /path/to/sample.exe -o ./unpacked
# With timeout (useful for Themida/VMProtect)
python scripts/run_unpacker.py /path/to/sample.exe -o ./unpacked --timeout 180
# Optional: max layers, confidence threshold
python scripts/run_unpacker.py /path/to/sample.exe -o ./unpacked --max-layers 5 --confidence 0.5
Example output:
Detected: aspack (confidence=0.9, method=sections)
Layer 1: packer=aspack -> ok
Final output: /path/to/unpacked/aspack/NotePad_aspack.unpacked.aspack.exe
If unpacking fails, you'll see the error (e.g. "Not a valid PE file" for PE32+, "Unpacking timed out", or "Generic unpacker stub").
Validating Unpacking: Why Proof Matters
Unpacking can fail silently (wrong OEP, truncated dump) or succeed but leave the file still packed (e.g. multi-layer). To check and validate correct unpack you need to:
Confirm format — Unpacked file is still valid PE/ELF (magic, structure).
Compare size — Unpacked is typically larger than packed (compression removed).
Compare entropy — Packed/compressed data has higher entropy ; unpacked code and data usually have lower entropy and more structure.
Re-run detection — Unpacked file should no longer be detected as the same packer (or not packed at all).
Inspect strings and metadata — More readable APIs, URLs, paths, and lower "obfuscated" hint suggest real code is visible.
The two tools below give you reproducible, static proof without running the sample.
Validation with String Analyzer
A Practical Guide to String Analyzer: Extract and Analyze Strings from Binaries (Without the…
Turn executables, memory dumps, and disk images into actionable intelligence in minutes — with one Python tool and zero…
medium.com
String Analyzer extracts printable strings, classifies them (URLs, IPs, Windows APIs, DLLs, obfuscation, etc.), and computes file entropy . High entropy + few "useful" patterns triggers a "likely obfuscated/packed" note — so before/after unpacking you should see entropy drop and often more APIs/DLLs/readable strings.
Install (if needed): From /home/andrey/git_project/String-Analyzer : pip install -e .
# or: python3 -m string_analyzer --help
Real example — ASPack packed vs unpacked (NotePad):
Packed: string-analyzer /home/andrey/git_project/Unpacker/samples_by_packer/upx/Cs2Loader_upx.exe
Filtered results saved -> /home/andrey/git_project/Unpacker/samples_by_packer/upx/Cs2Loader_upx_strings.txt
cat /home/andrey/git_project/Unpacker/samples_by_packer/upx/Cs2Loader_upx_strings.txt
File Entropy: 7.86
### CMD COMMANDS:
- exit
### DLLS:
- ADVAPI32.dll
- CRYPT32.dll
- GDI32.dll
- KERNEL32.DLL
- OLEAUT32.dll
- SHELL32.dll
- USER32.dll
- USERENV.dll
- VERSION.dll
- WINHTTP.dll
- WININET.dll
- WS2_32.dll
- api-ms-win-crt-convert-l1-1-0.dll
- api-ms-win-crt-environment-l1-1-0.dll
- api-ms-win-crt-heap-l1-1-0.dll
- api-ms-win-crt-locale-l1-1-0.dll
- api-ms-win-crt-math-l1-1-0.dll
- api-ms-win-crt-private-l1-1-0.dll
- api-ms-win-crt-runtime-l1-1-0.dll
- api-ms-win-crt-stdio-l1-1-0.dll
- api-ms-win-crt-string-l1-1-0.dll
- api-ms-win-crt-time-l1-1-0.dll
- api-ms-win-crt-utility-l1-1-0.dll
- bcrypt.dll
- ncrypt.dll
- ole32.dll
### FILES:
- te.txt
### IPS:
- version="1.0.0.0"
### OBFUSCATED:
- WINHTTP.dll
- WinHttpOpen
- ahttps://a
- version="1.0.0.0"
### SUSPICIOUS KEYWORDS:
- !This program cannot be run in DOS mode.$
-