Analyzing Malware Behavior with Cuckoo Sandbox

When to Use

• A suspicious sample passed static analysis triage and requires behavioral observation in a controlled environment
• You need to capture network traffic, file drops, registry modifications, and API calls from a malware execution
• Determining the full infection chain including second-stage payload downloads and persistence mechanisms
• Generating behavioral signatures and YARA rules based on observed runtime activity
• Automated analysis of bulk malware samples requiring consistent reporting

Do not use when the sample is a known ransomware variant that may spread via network shares in a misconfigured sandbox; verify network isolation first.

Prerequisites

• Cuckoo Sandbox 3.x installed on a dedicated analysis server (Ubuntu 22.04 recommended)
• Guest VMs configured with Windows 10/11 snapshots (Cuckoo agent installed, snapshots taken at clean state)
• VirtualBox, KVM, or VMware configured as the Cuckoo virtualization backend
• Isolated network with InetSim or FakeNet-NG for simulating internet services
• Suricata or Snort integrated for network-level signature matching during analysis
• Sufficient disk space for PCAP captures and memory dumps (minimum 500 GB recommended)

Workflow

Step 1: Submit Sample to Cuckoo

Submit the malware sample for automated analysis:

bash
# Submit via command line
cuckoo submit /path/to/suspect.exe

# Submit with specific analysis timeout (300 seconds)
cuckoo submit --timeout 300 /path/to/suspect.exe

# Submit with specific VM and analysis package
cuckoo submit --machine win10_x64 --package exe --timeout 300 /path/to/suspect.exe

# Submit via REST API
curl -F "file=@suspect.exe" -F "timeout=300" -F "machine=win10_x64" \
  http://localhost:8090/tasks/create/file

# Submit URL for analysis
curl -F "url=http://malicious-site.com/payload" -F "timeout=300" \
  http://localhost:8090/tasks/create/url

# Check task status
curl http://localhost:8090/tasks/view/1 | jq '.task.status'

Step 2: Monitor Execution in Real-Time

Track the analysis progress and observe live behavior:

bash
# Watch Cuckoo analysis log
tail -f /opt/cuckoo/log/cuckoo.log

# Monitor analysis task status
cuckoo status

# Access Cuckoo web interface for live screenshots and process tree
# Navigate to http://localhost:8080/analysis/<task_id>/

Key behavioral events to watch during execution:

• Process creation chain (parent-child relationships)
• Network connection attempts to external IPs
• File drops in temporary directories or system folders
• Registry modifications to Run keys or service entries
• API calls related to encryption (CryptEncrypt), injection (WriteProcessMemory), or evasion

Step 3: Analyze Process Activity

Review the process tree and API call trace from the Cuckoo report:

python
# Parse Cuckoo JSON report programmatically
import json

with open("/opt/cuckoo/storage/analyses/1/reports/report.json") as f:
    report = json.load(f)

# Process tree analysis
for process in report["behavior"]["processes"]:
    pid = process["pid"]
    ppid = process["ppid"]
    name = process["process_name"]
    print(f"PID: {pid} PPID: {ppid} Name: {name}")

    # Extract suspicious API calls
    for call in process["calls"]:
        api = call["api"]
        if api in ["CreateRemoteThread", "VirtualAllocEx", "WriteProcessMemory",
                    "NtCreateThreadEx", "RegSetValueExA", "URLDownloadToFileA"]:
            args = {arg["name"]: arg["value"] for arg in call["arguments"]}
            print(f"  [!] {api}({args})")

Step 4: Review Network Activity

Examine network connections, DNS queries, and HTTP requests:

python
# Network analysis from Cuckoo report
network = report["network"]

# DNS resolutions
print("DNS Queries:")
for dns in network.get("dns", []):
    print(f"  {dns['request']} -> {dns.get('answers', [])}")

# HTTP requests
print("\nHTTP Requests:")
for http in network.get("http", []):
    print(f"  {http['method']} {http['uri']} (Host: {http['host']})")
    if http.get("body"):
        print(f"    Body: {http['body'][:200]}")

# TCP connections
print("\nTCP Connections:")
for tcp in network.get("tcp", []):
    print(f"  {tcp['src']}:{tcp['sport']} -> {tcp['dst']}:{tcp['dport']}")

# Extract PCAP for deeper Wireshark analysis
# PCAP location: /opt/cuckoo/storage/analyses/1/dump.pcap

Step 5: Examine File System and Registry Changes

Document persistence mechanisms and dropped files:

python
# File operations
print("Files Created/Modified:")
for f in report["behavior"].get("summary", {}).get("files", []):
    print(f"  {f}")

# Dropped files with hashes
print("\nDropped Files:")
for dropped in report.get("dropped", []):
    print(f"  Path: {dropped['filepath']}")
    print(f"  SHA-256: {dropped['sha256']}")
    print(f"  Size: {dropped['size']} bytes")
    print(f"  Type: {dropped['type']}")

# Registry modifications
print("\nRegistry Keys Modified:")
for key in report["behavior"].get("summary", {}).get("keys", []):
    print(f"  {key}")

Step 6: Review Signatures and Scoring

Check Cuckoo's behavioral signatures and threat scoring:

python
# Behavioral signatures triggered
print("Triggered Signatures:")
for sig in report.get("signatures", []):
    severity = sig["severity"]
    name = sig["name"]
    description = sig["description"]
    marker = "[!]" if severity >= 3 else "[*]"
    print(f"  {marker} [{severity}/5] {name}: {description}")
    for mark in sig.get("marks", []):
        if mark.get("call"):
            print(f"      API: {mark['call']['api']}")
        if mark.get("ioc"):
            print(f"      IOC: {mark['ioc']}")

# Overall score
score = report.get("info", {}).get("score", 0)
print(f"\nOverall Threat Score: {score}/10")

Step 7: Extract Memory Dump Artifacts

Analyze the full memory dump captured during execution:

bash
# Memory dump is saved at:
# /opt/cuckoo/storage/analyses/1/memory.dmp

# Use Volatility to analyze the memory dump
vol3 -f /opt/cuckoo/storage/analyses/1/memory.dmp windows.pslist
vol3 -f /opt/cuckoo/storage/analyses/1/memory.dmp windows.malfind
vol3 -f /opt/cuckoo/storage/analyses/1/memory.dmp windows.netscan

Key Concepts

| Term | Definition | |------|------------| | Dynamic Analysis | Executing malware in a controlled environment to observe runtime behavior including system calls, network activity, and file operations | | Sandbox Evasion | Techniques malware uses to detect virtual/sandbox environments and alter behavior to avoid analysis (sleep timers, VM checks, user interaction checks) | | API Hooking | Cuckoo's method of intercepting Windows API calls made by the malware to log function names, parameters, and return values | | InetSim | Internet services simulation tool that responds to malware network requests (HTTP, DNS, SMTP) within the isolated analysis network | | Process Injection | Malware technique of injecting code into legitimate processes; detected by monitoring VirtualAllocEx and WriteProcessMemory API sequences | | Behavioral Signature | Rule-based detection matching specific sequences of API calls, file operations, or network activity to known malware behaviors | | Analysis Package | Cuckoo module defining how to execute a specific file type (exe, dll, pdf, doc) within the guest VM for proper behavioral capture |

Tools & Systems

• Cuckoo Sandbox: Open-source automated malware analysis system providing behavioral reports, network captures, and memory dumps
• InetSim: Internet services simulation suite providing fake HTTP, DNS, SMTP, and other services for isolated malware analysis networks
• FakeNet-NG: FLARE team's network simulation tool that intercepts and redirects all network traffic for analysis
• Suricata: Network IDS/IPS integrated with Cuckoo for real-time signature-based detection of malicious network traffic
• Volatility: Memory forensics framework used to analyze memory dumps captured during Cuckoo analysis

Common Scenarios

Scenario: Analyzing a Multi-Stage Dropper

Context: Static analysis reveals a packed executable with minimal imports and high entropy. The sample needs sandbox execution to observe unpacking, payload delivery, and C2 establishment.

Approach:

1. Submit sample to Cuckoo with extended timeout (600 seconds) to capture slow-acting behavior
2. Review process tree for child process creation (dropper spawning payload processes)
3. Identify dropped files in %TEMP%, %APPDATA%, or system directories
4. Extract dropped files and compute hashes for separate analysis
5. Map network connections to identify C2 infrastructure contacted after initial execution
6. Check for persistence mechanisms (Run keys, scheduled tasks, services) in registry modifications
7. Compare behavioral signatures against known malware families

Pitfalls:

• Using insufficient analysis timeout causing the sandbox to terminate before second-stage payload executes
• Not configuring InetSim to respond to DNS and HTTP requests, preventing the malware from progressing past C2 check-in
• Ignoring sandbox evasion detections; if the sample exits immediately, it may be detecting the virtual environment
• Not analyzing dropped files separately; the initial dropper may be less interesting than the final payload

Output Format

DYNAMIC ANALYSIS REPORT - CUCKOO SANDBOX
==========================================
Task ID:          1547
Sample:           suspect.exe (SHA-256: e3b0c44298fc1c149afbf4c8996fb924...)
Analysis Time:    300 seconds
VM:               win10_x64 (Windows 10 21H2)
Score:            8.5/10

PROCESS TREE
suspect.exe (PID: 2184)
  └── cmd.exe (PID: 3456)
      └── powershell.exe (PID: 4012)
          └── svchost_fake.exe (PID: 4568)

FILE SYSTEM ACTIVITY
[CREATED]  C:\Users\Admin\AppData\Local\Temp\payload.dll
[CREATED]  C:\Windows\System32\svchost_fake.exe
[MODIFIED] C:\Windows\System32\drivers\etc\hosts

REGISTRY MODIFICATIONS
[SET] HKCU\Software\Microsoft\Windows\CurrentVersion\Run\WindowsUpdate = "C:\Windows\System32\svchost_fake.exe"
[SET] HKLM\SYSTEM\CurrentControlSet\Services\FakeService\ImagePath = "C:\Windows\System32\svchost_fake.exe"

NETWORK ACTIVITY
DNS:    update.malicious[.]com -> 185.220.101.42
HTTP:   POST hxxps://185.220.101[.]42/gate.php (beacon)
TCP:    10.0.2.15:49152 -> 185.220.101.42:443 (237 connections)

BEHAVIORAL SIGNATURES
[!] [4/5] injection_createremotethread: Injects code into remote process
[!] [4/5] persistence_autorun: Modifies Run registry key for persistence
[!] [3/5] network_cnc_http: Performs HTTP C2 communication
[*] [2/5] antiav_detectfile: Checks for antivirus product files

DROPPED FILES
payload.dll    SHA-256: abc123... Size: 98304  Type: PE32 DLL
svchost_fake.exe SHA-256: def456... Size: 184320 Type: PE32 EXE