Implementation:Tencent Ncnn ModelWriter
| Knowledge Sources | |
|---|---|
| Domains | Model_Serialization, Model_Optimization |
| Last Updated | 2026-02-09 19:00 GMT |
Overview
Core model serialization and analysis engine used by ncnn tools such as ncnnoptimize, capable of loading an ncnn model, performing shape inference, estimating memory footprint, and writing the model back out in ncnn's param/bin format with optional fp16 weight storage.
Description
The file defines three key classes:
MemoryFootprintAllocator is a custom allocator that wraps ncnn's fastMalloc/fastFree and tracks current memory usage and peak memory footprint using a bookkeeper map, protected by a mutex for thread safety.
CustomLayer extends ncnn::Layer to handle unrecognized layer types by storing their parameter dictionaries verbatim and writing them back unchanged during serialization.
The central ModelWriter class extends ncnn::Net and exposes direct access to the network's internal blobs and layers vectors via references from mutable_blobs()/mutable_layers(). It supports:
- Model cutting - extracting a sub-graph between named start/end layers via set_cutparam
- Shape inference - running a dummy forward pass through an Extractor to resolve all blob shapes
- Memory footprint estimation - using MemoryFootprintAllocator to track peak memory during inference
- Model saving - iterating over all layers and writing each one's type, name, blob connections, and layer-specific parameters to the param file, while writing weight data to the bin file
Weight serialization uses fwrite_weight_tag_data which can optionally convert fp32 weights to fp16 format (storage_type=1). The file includes headers for nearly all ncnn layer types so it can directly cast layer pointers and access their weight data fields (e.g., convolution->weight_data, batchnorm->slope_data) for serialization. A PRNG seeded with ncnn's magic number 7767517 is used when gen_random_weight is enabled to generate synthetic weights for testing.
Usage
This is a core infrastructure component for ncnn's tooling. It is the base class for NetOptimize in ncnnoptimize.cpp and provides all the model I/O, analysis, and serialization functionality that the optimization pipeline depends on. Any tool that needs to load, modify, and re-save ncnn models relies on this class.
Code Reference
Source Location
- Repository: Tencent_Ncnn
- File: tools/modelwriter.h
- Lines: 1-2597
Signature
class MemoryFootprintAllocator : public ncnn::Allocator {
public:
MemoryFootprintAllocator();
virtual void* fastMalloc(size_t size);
virtual void fastFree(void* ptr);
int current_memory_usage;
int memory_footprint;
};
class CustomLayer : public ncnn::Layer {
public:
virtual int load_param(const ncnn::ParamDict& pd);
void write_param(FILE* pp);
};
class ModelWriter : public ncnn::Net {
public:
ModelWriter();
virtual ncnn::Layer* create_custom_layer(const char* type);
std::vector<ncnn::Blob>& blobs;
std::vector<ncnn::Layer*>& layers;
bool has_custom_layer;
int storage_type; // 0=fp32, 1=fp16
int gen_random_weight;
int cutstart, cutend; // -1=no cut
int set_cutparam(const char* cutstartname, const char* cutendname);
int shape_inference();
int estimate_memory_footprint();
int fprintf_param_int_array(int id, const ncnn::Mat& m, FILE* pp);
int fprintf_param_float_array(int id, const ncnn::Mat& m, FILE* pp);
int fwrite_weight_tag_data(const ncnn::Mat& data, FILE* bp, float a = -1.2f, float b = 1.2f);
int fwrite_weight_data(const ncnn::Mat& data, FILE* bp, float a = -1.2f, float b = 1.2f);
int save(const char* parampath, const char* binpath);
};
Import
// Header-only library, included by ncnnoptimize and similar tools
#include "modelwriter.h"
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| parampath | const char* | Yes | Path to ncnn .param file to load or save |
| binpath | const char* | Yes | Path to ncnn .bin file to load or save |
| storage_type | int | No | Weight storage type: 0=fp32 (default), 1=fp16 |
| cutstartname | const char* | No | Layer name to start the sub-graph cut |
| cutendname | const char* | No | Layer name to end the sub-graph cut |
Outputs
| Name | Type | Description |
|---|---|---|
| param file | text file | ncnn parameter file with layer definitions and parameters |
| bin file | binary file | ncnn binary weight file, optionally with fp16 storage |
| memory_footprint | int | Peak memory usage in bytes (from estimate_memory_footprint) |
Usage Examples
Load, Optimize, and Save a Model
ModelWriter writer;
writer.storage_type = 1; // fp16 output
writer.load_param("model.param");
writer.load_model("model.bin");
writer.shape_inference();
writer.save("optimized.param", "optimized.bin");
Estimate Memory Footprint
ModelWriter writer;
writer.load_param("model.param");
writer.load_model("model.bin");
int peak_mem = writer.estimate_memory_footprint();
fprintf(stderr, "Peak memory: %d bytes\n", peak_mem);