Implementation:Tencent Ncnn Layer Base Class

Knowledge Sources	Tencent_Ncnn
Domains	Neural Network Inference, Operator Framework, Plugin Architecture
Last Updated	2026-02-09 19:00 GMT

Overview

Base class for all neural network layers in ncnn, defining the virtual interface for parameter loading, weight loading, pipeline management, and forward inference across both CPU and Vulkan backends.

Description

The Layer base class is the foundational abstraction for ncnn's operator plugin system. Every one of ncnn's 100+ built-in operators (Convolution, ReLU, Pooling, Softmax, etc.) inherits from this class, and custom user-defined layers also extend it.

The class defines a virtual interface with the following lifecycle methods:

• load_param(const ParamDict& pd) -- Parse layer-specific parameters from the parameter dictionary during model loading
• load_model(const ModelBin& mb) -- Load layer-specific weight data from the model binary file
• create_pipeline(const Option& opt) -- Perform implementation-specific setup (e.g., weight packing, kernel transformation)
• destroy_pipeline(const Option& opt) -- Clean up implementation-specific resources

The inference interface provides two patterns:

• forward() -- Out-of-place inference: reads from bottom blobs and writes to separate top blobs
• forward_inplace() -- In-place inference: reads and writes the same blob, saving memory

Both patterns have single-blob variants (one input, one output) and multi-blob variants (vector of inputs, vector of outputs). The default forward() implementation clones inputs and delegates to forward_inplace() when the layer supports inplace operation.

Capability flags declared as boolean members control how the framework interacts with each layer:

• one_blob_only -- Layer accepts exactly one input and produces one output
• support_inplace -- Layer can perform inference without allocating separate output
• support_packing -- Layer accepts packed element storage (e.g., 4 or 8 floats per element)
• support_vulkan -- Layer has a Vulkan GPU compute implementation
• support_fp16_storage / support_bf16_storage / support_int8_storage -- Precision support flags

The layer factory system uses auto-generated registry tables and runtime CPU detection to select the best SIMD-optimized implementation. The internal Layer_final wrapper class holds both a CPU layer and an optional Vulkan layer, delegating all virtual calls to the appropriate backend. The create_layer_cpu() function checks architecture-specific registries (AVX-512, FMA, AVX, LASX, LSX, MSA, xtheadvector, RVV) with fallback to the generic registry.

Macros DEFINE_LAYER_CREATOR and DEFINE_LAYER_DESTROYER generate boilerplate factory functions for custom layers.

Usage

Use this class as the base when implementing custom neural network operators. Subclass Layer, override the lifecycle and inference methods, set the appropriate capability flags, and register the layer with the Net using the custom layer registration API. All built-in ncnn layers follow this same pattern.

Code Reference

Source Location

• Repository: Tencent_Ncnn
• Header: src/layer.h (212 lines)
• Implementation: src/layer.cpp (537 lines)

Signature

namespace ncnn {

class NCNN_EXPORT Layer
{
public:
    Layer();
    virtual ~Layer();

    // Load layer specific parameter from parsed dict (return 0 if success)
    virtual int load_param(const ParamDict& pd);

    // Load layer specific weight data from model binary (return 0 if success)
    virtual int load_model(const ModelBin& mb);

    // Layer implementation specific setup (return 0 if success)
    virtual int create_pipeline(const Option& opt);

    // Layer implementation specific clean (return 0 if success)
    virtual int destroy_pipeline(const Option& opt);

public:
    bool one_blob_only;
    bool support_inplace;
    bool support_vulkan;
    bool support_packing;
    bool support_bf16_storage;
    bool support_fp16_storage;
    bool support_int8_storage;
    bool support_tensor_storage;
    int featmask;

public:
    // Inference (return 0 if success)
    virtual int forward(const std::vector<Mat>& bottom_blobs,
                        std::vector<Mat>& top_blobs,
                        const Option& opt) const;
    virtual int forward(const Mat& bottom_blob,
                        Mat& top_blob,
                        const Option& opt) const;

    // Inplace inference (return 0 if success)
    virtual int forward_inplace(std::vector<Mat>& bottom_top_blobs,
                                const Option& opt) const;
    virtual int forward_inplace(Mat& bottom_top_blob,
                                const Option& opt) const;

public:
    void* userdata;
    int typeindex;
    std::string type;
    std::string name;
    std::vector<int> bottoms;
    std::vector<int> tops;
    std::vector<Mat> bottom_shapes;
    std::vector<Mat> top_shapes;
};

// Factory functions
typedef Layer* (*layer_creator_func)(void*);
typedef void (*layer_destroyer_func)(Layer*, void*);

NCNN_EXPORT int layer_to_index(const char* type);
NCNN_EXPORT Layer* create_layer(const char* type);
NCNN_EXPORT Layer* create_layer(int index);
NCNN_EXPORT Layer* create_layer_cpu(const char* type);

#define DEFINE_LAYER_CREATOR(name) \
    ::ncnn::Layer* name##_layer_creator(void* /*userdata*/) \
    { return new name; }

#define DEFINE_LAYER_DESTROYER(name) \
    void name##_layer_destroyer(::ncnn::Layer* layer, void* /*userdata*/) \
    { delete layer; }

} // namespace ncnn

Import

#include "ncnn/layer.h"

I/O Contract

Inputs

Name	Type	Required	Description
pd	const ParamDict&	Yes (load_param)	Dictionary of layer parameters parsed from the .param file
mb	const ModelBin&	Yes (load_model)	Binary reader for loading layer weight data from the .bin file
opt	const Option&	Yes	Runtime options (thread count, allocator, precision flags)
bottom_blobs	const std::vector<Mat>&	Yes (forward)	Input tensors for multi-blob forward
bottom_blob	const Mat&	Yes (forward)	Single input tensor for one-blob forward

Outputs

Name	Type	Description
return	int	0 on success, -1 on not implemented, -100 on memory allocation failure
top_blobs	std::vector<Mat>&	Output tensors for multi-blob forward
top_blob	Mat&	Single output tensor for one-blob forward

Usage Examples

Implementing a Custom Layer

#include "ncnn/layer.h"

class MyCustomLayer : public ncnn::Layer
{
public:
    MyCustomLayer()
    {
        one_blob_only = true;
        support_inplace = true;
    }

    virtual int load_param(const ncnn::ParamDict& pd)
    {
        scale = pd.get(0, 1.f);
        return 0;
    }

    virtual int forward_inplace(ncnn::Mat& bottom_top_blob,
                                const ncnn::Option& opt) const
    {
        int size = bottom_top_blob.total();
        float* ptr = bottom_top_blob;
        for (int i = 0; i < size; i++)
        {
            ptr[i] *= scale;
        }
        return 0;
    }

public:
    float scale;
};

DEFINE_LAYER_CREATOR(MyCustomLayer)

Registering a Custom Layer with Net

#include "ncnn/net.h"

ncnn::Net net;
net.register_custom_layer("MyCustomLayer", MyCustomLayer_layer_creator);
net.load_param("model.param");
net.load_model("model.bin");