Principle:Protectai Modelscan Model Security Scanning
| Knowledge Sources | |
|---|---|
| Domains | ML_Security, Supply_Chain_Security |
| Last Updated | 2026-02-14 12:00 GMT |
Overview
A security analysis technique that inspects serialized machine learning model files for embedded unsafe operations, malicious code, or dangerous imports before the model is loaded into memory.
Description
Model Security Scanning addresses a critical vulnerability in the ML supply chain: serialized model files (pickle, HDF5, SavedModel, etc.) can contain arbitrary code that executes upon deserialization. Attackers exploit formats like Python's pickle protocol, which supports arbitrary object construction via opcodes such as GLOBAL, INST, and STACK_GLOBAL, to embed calls to dangerous modules (e.g., os, subprocess, builtins.exec). Similarly, TensorFlow SavedModel files can include unsafe graph operations (ReadFile, WriteFile), and Keras models can embed Lambda layers containing arbitrary Python code.
The scanning approach works by static analysis of serialized data rather than executing the model. For pickle-based formats, this means disassembling the bytecode to extract all referenced global imports and comparing them against a curated list of known-dangerous operations. For TensorFlow formats, it parses protocol buffer definitions to extract operation names. For Keras formats, it inspects model configuration JSON for Lambda layers.
Usage
Apply this principle when:
- Downloading pre-trained models from public repositories (HuggingFace Hub, PyTorch Hub)
- Receiving model files from untrusted sources
- Integrating model scanning into CI/CD or MLOps pipelines
- Auditing existing model artifacts for supply chain risks
This is a preventive security control that should be applied before any model deserialization occurs (before calling pickle.load(), torch.load(), tf.saved_model.load(), etc.).
Theoretical Basis
The security scanning technique relies on three key observations:
1. Serialization formats embed executable references:
Python's pickle protocol encodes object construction instructions as bytecode opcodes. The critical opcodes are:
GLOBAL - Push a global object (module.name) onto the stack
INST - Build a class instance (legacy opcode)
STACK_GLOBAL - Push a global found by name on the stack (protocol 4+)
2. Static analysis can extract these references without execution:
# Pseudo-code for pickle scanning
for opcode, arg, pos in disassemble(pickle_bytes):
if opcode in (GLOBAL, INST, STACK_GLOBAL):
module, operator = parse_reference(arg)
if (module, operator) in unsafe_globals:
report_issue(module, operator, severity)
3. Severity classification based on capability:
Detected operators are classified by the damage they can cause:
- CRITICAL: Code execution (os.system, subprocess.Popen, builtins.exec)
- HIGH: Network access (webbrowser.open, requests.api)
- MEDIUM: Suspicious but potentially legitimate operations
- LOW: Low-risk anomalies
For TensorFlow/Keras formats, the analysis inspects protocol buffer structures and JSON configuration rather than pickle bytecode, but follows the same principle of static extraction and severity classification.