Workflow:Protectai Modelscan CLI Model Scanning
| Knowledge Sources | |
|---|---|
| Domains | ML_Security, Model_Scanning, CLI_Tools |
| Last Updated | 2026-02-14 12:00 GMT |
Overview
End-to-end process for scanning machine learning model files via the ModelScan command-line interface to detect model serialization attacks.
Description
This workflow describes the standard procedure for using ModelScan's CLI to scan ML model files for unsafe embedded code. ModelScan reads model files byte-by-byte without executing them, looking for code signatures that indicate malicious payloads such as credential theft, data exfiltration, or arbitrary code execution. The tool supports Pickle-based formats (PyTorch, Sklearn, XGBoost, joblib, dill), TensorFlow SavedModel (Protocol Buffers), and Keras formats (H5 and .keras). Each detected issue is classified by severity: CRITICAL, HIGH, MEDIUM, or LOW. The CLI produces either a human-readable console report or a machine-readable JSON report.
Usage
Execute this workflow when you need to verify the safety of a model file before loading it into your ML pipeline. This applies when downloading models from public repositories (e.g., Hugging Face), receiving models from external teams, or validating models at any stage of an MLOps pipeline (pre-training, post-training, pre-deployment).
Execution Steps
Step 1: Install ModelScan
Install the ModelScan package from PyPI into your Python environment. The base installation supports Pickle-derived formats (PyTorch, Sklearn, XGBoost, joblib, dill) and NumPy. For TensorFlow SavedModel or Keras H5 scanning, install with the appropriate extras to pull in the required binary dependencies (tensorflow and h5py respectively).
Key considerations:
- Python 3.9 through 3.12 is supported
- The base install covers Pickle, PyTorch, NumPy, and classical ML library formats
- TensorFlow and H5 scanning require separate extras installation
- The package registers a console script entry point named modelscan
Step 2: Prepare Scan Target
Identify the model file or directory to scan. ModelScan accepts a single file path or a directory path. When given a directory, it recursively discovers all files and attempts to scan each one. The tool automatically detects the model format based on file extension through its middleware pipeline, matching extensions like .pkl, .pt, .h5, .keras, .pb, and others to the appropriate scanner.
Key considerations:
- Supported extensions include .pkl, .pickle, .joblib, .dill, .dat, .data, .bin, .pt, .pth, .ckpt, .npy, .h5, .keras, .pb
- Zip archives (.zip, .npz) are automatically opened and their contents scanned
- Nested zip files are not supported and will be reported as errors
- Files with unsupported extensions are skipped and reported in the output
Step 3: Configure Settings (Optional)
Optionally create or provide a custom settings file in TOML format to override default scan behavior. The settings file controls which scanners are enabled, what file extensions they handle, and which operator patterns are considered unsafe at each severity level. Generate a default settings file using the create-settings-file subcommand, then modify it as needed.
Key considerations:
- The default settings file location is ./modelscan-settings.toml in the current working directory
- Settings control scanner enablement, supported extensions, unsafe globals lists, and reporting modules
- Custom reporting modules can be configured for integration with external systems
- The unsafe globals dictionary maps Python module names to severity levels (e.g., os.* is CRITICAL, webbrowser.* is HIGH)
Step 4: Execute Scan
Run the modelscan command with the -p flag pointing to the target model file or directory. The scan engine initializes all enabled scanners and the middleware pipeline, then iterates over each discovered model file. For each file, the middleware tags it with its format based on extension. Each scanner checks whether it handles that format and, if so, performs its analysis: Pickle-based scanners disassemble bytecode to extract global imports, TensorFlow scanners parse Protocol Buffers for unsafe operations, and Keras scanners inspect model configs for Lambda layers.
Key considerations:
- The scan reads files byte-by-byte without executing any model code
- Pickle scanning uses pickletools.genops() to disassemble opcodes and extract GLOBAL, INST, and STACK_GLOBAL operations
- TensorFlow scanning parses SavedModel protobuf to extract operation names from graph definitions
- Keras scanning parses JSON model configs to detect Lambda layers
- Multiple scanners may evaluate the same file if it matches multiple formats
Step 5: Review Results
Examine the scan output to determine whether any unsafe operators were found. The default console report groups issues by severity and lists each detected unsafe operator with its module, operator name, and source location. Alternatively, use JSON output format for programmatic consumption. The CLI exit code indicates the overall result: 0 for clean, 1 for issues found, 2 for scan errors, 3 for no supported files, and 4 for usage errors.
Key considerations:
- Issues are grouped by severity: CRITICAL, HIGH, MEDIUM, LOW
- Each issue identifies the unsafe module and operator (e.g., os.system, builtins.eval)
- The --show-skipped flag reveals files that were not scanned and why
- JSON output can be written to a file using the -o flag for pipeline integration
- Exit codes enable integration with CI/CD systems for automated pass/fail decisions
Step 6: Triage and Remediate
For each reported issue, evaluate whether the embedded operator represents a genuine security threat or a legitimate (though risky) usage pattern. Contact the model author to understand the purpose of any flagged code. In production pipelines, integrate the scan at three points: before loading pre-trained models, after training new models, and before deploying models to endpoints.
Key considerations:
- Some models legitimately embed code for reproducibility, but this opens attack surface
- CRITICAL severity items (e.g., os.*, subprocess.*, builtins.eval) are almost always dangerous
- HIGH severity items (e.g., webbrowser.*, httplib.*) indicate network access capabilities
- Models that fail scanning should not be loaded until verified safe