Implementation:LaurentMazare Tch rs Tensor File CLI Tool
| Knowledge Sources | |
|---|---|
| Domains | CLI Utility, Tensor Serialization, Data Conversion |
| Last Updated | 2026-02-08 00:00 GMT |
Overview
A command-line utility for listing the contents of tensor files and converting between tensor file formats including npy, npz, safetensors, ot, and bin/zip.
Description
This CLI tool provides two subcommands: ls and cp. The ls subcommand inspects one or more tensor files and prints the tensor names, shapes, and types. It supports five file formats identified by extension: .npy (single NumPy tensor via read_npy), .npz (multiple NumPy tensors via read_npz), .safetensors (HuggingFace safetensors via read_safetensors), .ot (tch native format via load_multi), and .bin/.zip (PyTorch pickle format via loadz_multi).
The cp subcommand converts tensors between formats by reading from a source file and writing to a destination file. It supports npz, safetensors, ot, and bin/zip as source formats, and npz, safetensors, and ot as destination formats. During conversion, it prints the source tensor metadata.
Format detection is based entirely on file extension matching. The tool uses the anyhow crate for error handling and provides descriptive error messages for unrecognized file extensions or incorrect usage.
Usage
Use this tool when you need to inspect tensor file contents (names, shapes, dtypes) or convert model weights between formats. It is especially useful for converting pre-trained weights from Python (npz/safetensors) to the tch-rs native .ot format for loading in Rust applications.
Code Reference
Source Location
- Repository: LaurentMazare_Tch_rs
- File: examples/tensor-tools.rs
- Lines: 1-79
Signature
pub fn main() -> Result<()>
Import
use anyhow::{bail, ensure, Result};
I/O Contract
| Subcommand | Arguments | Description |
|---|---|---|
| ls | file1 [file2 ...] | List tensor names and shapes from one or more files |
| cp | src dst | Convert tensors from source format to destination format |
| Supported Format | Extension | Read (ls/cp src) | Write (cp dst) |
|---|---|---|---|
| NumPy single | .npy | Yes (ls only) | No |
| NumPy archive | .npz | Yes | Yes |
| Safetensors | .safetensors | Yes | Yes |
| tch native | .ot | Yes | Yes |
| PyTorch pickle | .bin, .zip | Yes | No |
Usage Examples
// List contents of tensor files (CLI usage):
// cargo run --example tensor-tools ls model.npy weights.npz checkpoint.ot
// Convert npz weights to tch-rs native .ot format:
// cargo run --example tensor-tools cp yolo-v3.ot.npz yolo-v3.ot
// Convert safetensors to npz:
// cargo run --example tensor-tools cp model.safetensors model.npz
// Programmatic equivalent of ls:
let tensors = tch::Tensor::read_npz("weights.npz")?;
for (name, tensor) in tensors.iter() {
println!("{}: {:?}", name, tensor);
}
// Programmatic equivalent of cp (npz -> ot):
let tensors = tch::Tensor::read_npz("src.npz")?;
tch::Tensor::save_multi(&tensors, "dst.ot")?;