Principle:Trailofbits Fickling File Property Discovery
| Knowledge Sources | |
|---|---|
| Domains | Security, File_Format, Supply_Chain |
| Last Updated | 2026-02-14 14:00 GMT |
Overview
A file introspection technique that discovers structural properties of PyTorch files (ZIP membership, tar structure, pickle validity, numpy headers) without fully parsing or executing them.
Description
File Property Discovery examines a binary file at the structural level to determine what it contains. It checks for:
- PyTorch ZIP: Uses PyTorch's own _is_zipfile magic number check
- Standard ZIP: Uses Python's zipfile.is_zipfile
- Tar archive: Uses tarfile.is_tarfile
- Valid pickle: Attempts to parse with Pickled.load
- NumPy array: Checks for NumPy magic bytes and header format
- ZIP contents: Looks for data.pkl, constants.pkl, version, model.json, attributes.pkl
This produces a boolean property dictionary that is the input for format identification. By separating property discovery from format classification, the system supports extensibility and debugging.
Usage
Use this as the first step in PyTorch file format identification. The properties dictionary enables both automated format classification and manual investigation of unusual files.
Theoretical Basis
# Pseudocode: Multi-probe file inspection
properties = {
"is_torch_zip": check_torch_magic_number(file),
"is_tar": check_tar_header(file),
"is_valid_pickle": try_parse_pickle(file),
"is_numpy": check_numpy_magic(file),
"has_data_pkl": check_zip_contains(file, "data.pkl"),
"has_constants_pkl": check_zip_contains(file, "constants.pkl"),
# ... more properties
}