Workflow:Trailofbits Fickling Pickle Safety Analysis
| Knowledge Sources | |
|---|---|
| Domains | Static_Analysis, ML_Security, Supply_Chain_Security |
| Last Updated | 2026-02-14 13:00 GMT |
Overview
End-to-end process for statically analyzing Python pickle files to detect malicious payloads using Fickling's AST-based safety analysis framework.
Description
This workflow performs static security analysis on pickle-serialized files without executing their contents. Fickling interprets the pickle bytecode using a symbolic virtual machine, converts it to a Python AST, and then runs multiple analysis passes to detect dangerous patterns. The analysis framework classifies findings using a six-level severity scale from LIKELY_SAFE to OVERTLY_MALICIOUS, checking for unsafe imports, suspicious function calls (eval, exec, compile, open), unused variables that indicate side-effect-only code, malformed opcode sequences, and non-standard library imports.
The workflow supports both a CLI interface (fickling --check-safety) and a programmatic Python API (fickling.load(), fickling.is_likely_safe(), check_safety()). Analysis results can be output as structured JSON for integration into CI/CD pipelines and automated scanning systems.
Usage
Execute this workflow when you need to vet pickle files before loading them, integrate pickle scanning into a CI/CD pipeline, audit model files from untrusted sources, or build automated model scanning infrastructure. It is suitable for pre-deployment validation, security auditing, and supply chain security gates.
Execution Steps
Step 1: Load Pickle Bytecode
Read the pickle file and parse it into Fickling's internal representation. The parser handles both single pickle streams and stacked pickle files (multiple pickle objects concatenated, as used by some ML frameworks). Invalid or malformed opcodes are flagged immediately.
Key considerations:
- Stacked pickle files (common in PyTorch) are automatically detected and each sub-pickle is analyzed
- Parsing errors themselves may indicate malicious tampering
- No code is executed during parsing; the pickle VM runs symbolically
Step 2: Convert to Python AST
Run Fickling's symbolic pickle interpreter to convert the opcode stream into a Python abstract syntax tree. The interpreter simulates the pickle virtual machine's stack operations, memo table, and object construction without executing any code.
Key considerations:
- The interpreter tracks variable assignments, imports, and function calls
- Interpretation errors (malformed opcode sequences) are recorded as potential tampering indicators
- The resulting AST represents what the pickle would do if deserialized normally
Step 3: Run Analysis Passes
Execute the full suite of analysis passes against the generated AST. Each analysis focuses on a specific class of threat:
Analysis passes include:
- DuplicateProtoAnalysis - detects duplicate PROTO opcodes indicating tampering
- MisplacedProtoAnalysis - detects PROTO opcodes in wrong position
- InvalidOpcode - detects invalid or corrupted opcode sequences
- InterpretationErrorAnalysis - detects malformed opcode sequences
- NonStandardImports - flags imports outside Python standard library
- UnsafeImportsML - checks against known dangerous modules (os, subprocess, builtins, etc.)
- BadCalls - detects direct calls to exec, eval, compile, open
- OvertlyBadEvals - detects suspicious function calls and side effects
- UnsafeImports - flags imports from known attack vectors
- UnusedVariables - detects variables assigned but never used (common in exploit payloads)
- MLAllowlist - validates imports against curated ML-safe import list
Step 4: Classify Severity
Aggregate results from all analysis passes and determine the overall severity level. The highest severity finding determines the file's classification.
Severity levels:
- LIKELY_SAFE (0) - No unsafe operations discovered
- POSSIBLY_UNSAFE (1) - Minor concerns
- SUSPICIOUS (2) - Unusual patterns detected
- LIKELY_UNSAFE (3) - Dangerous patterns found
- LIKELY_OVERTLY_MALICIOUS (4) - Strong indicators of malice
- OVERTLY_MALICIOUS (5) - Confirmed malicious patterns
Step 5: Output Results
Present the analysis results in the requested format. The CLI outputs human-readable text to stderr and optionally writes structured JSON. The API returns an AnalysisResults object with severity, detailed messages, and per-analysis breakdowns.
Key considerations:
- JSON output supports CI/CD integration and automated decision-making
- The to_dict() method provides structured data including severity, analysis messages, and trigger details
- Results can be filtered by minimum verbosity level