Implementation:Tencent Ncnn Piper TTS Example
| Knowledge Sources | |
|---|---|
| Domains | Audio, Text_To_Speech |
| Last Updated | 2026-02-09 19:00 GMT |
Overview
Concrete tool for text-to-speech synthesis inference using Piper TTS (VITS architecture) with ncnn.
Description
This example implements a complete text-to-speech pipeline using the Piper TTS system, a VITS-based generative speech model. It includes three custom ncnn layer implementations required by the VITS architecture: relative_embeddings_k_module and relative_embeddings_v_module for windowed relative position embeddings in the attention mechanism, and piecewise_rational_quadratic_transform_module for the normalizing flow component. The pipeline operates in five stages: (1) Phonemization -- a simple grapheme-to-phoneme function converts input English text to phoneme IDs using a precompiled binary dictionary (en-word_id.bin); (2) Encoder (enc_p) -- encodes the phoneme sequence into hidden representations, mean, and log-scale parameters; (3) Speaker embedding (emb_g) -- selects a speaker embedding by ID (supports 904 speakers); (4) Duration predictor (dp) -- predicts phoneme durations using a stochastic duration predictor with the flow-based transform; (5) Flow + Decoder -- the flow model converts latent representations to mel-spectrogram features, and the decoder (dec) generates raw audio waveform. Output is normalized, clipped, and saved as a 16-bit PCM WAV file at 22050 Hz sample rate. Models are converted from Piper checkpoints via PNNX.
Usage
Use this example to convert English text to speech audio with selectable speaker identity. It takes a text string, speaker ID (0-903), and output WAV path as arguments.
Code Reference
Source Location
- Repository: Tencent_Ncnn
- File: examples/piper.cpp
- Lines: 1-739
Signature
static void simple_phonemize(const char* text, std::vector<int>& sequence_ids);
static void path_attention(const ncnn::Mat& logw, const ncnn::Mat& m_p, const ncnn::Mat& logs_p, float noise_scale, float length_scale, ncnn::Mat& z_p);
static int tts_piper(const char* text, int speaker_id, std::vector<short>& pcm);
static void save_pcm_to_wav(const char* path, const short* pcm, int num_samples, int sample_rate);
int main(int argc, char** argv);
Import
#include "layer.h"
#include "mat.h"
#include "net.h"
#include <ctype.h>
#include <stdio.h>
#include <map>
#include <vector>
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| sentences | const char* (argv[1]) | Yes | Input English text to synthesize |
| speaker_id | int (argv[2]) | Yes | Speaker identity index (0 to 903) |
| out_path | const char* (argv[3]) | Yes | Output WAV file path |
Outputs
| Name | Type | Description |
|---|---|---|
| WAV file | 16-bit PCM | Mono audio at 22050 Hz sample rate written to specified output path |
Usage Examples
Running the Example
./piper "Hello World" 0 out.wav
./piper "Happy New Year" 123 out.wav
Key Code Pattern
// 1. Phonemize text to sequence IDs
std::vector<int> sequence_ids;
simple_phonemize(text, sequence_ids);
// 2. Encoder
ncnn::Net enc_p;
enc_p.register_custom_layer("...relative_embeddings_k_module", relative_embeddings_k_module_layer_creator);
enc_p.register_custom_layer("...relative_embeddings_v_module", relative_embeddings_v_module_layer_creator);
enc_p.load_param("en_enc_p.ncnn.param");
enc_p.load_model("en_enc_p.ncnn.bin");
ncnn::Extractor ex = enc_p.create_extractor();
ex.input("in0", sequence);
ex.extract("out0", x);
ex.extract("out1", m_p);
ex.extract("out2", logs_p);
// 3. Speaker embedding
ncnn::Net emb_g;
emb_g.load_param("en_emb_g.ncnn.param");
// ... select speaker by ID -> g
// 4. Duration predictor (with flow transform)
ncnn::Net dp;
dp.register_custom_layer("...piecewise_rational_quadratic_transform_module", ...);
dp.load_param("en_dp.ncnn.param");
// ... predict durations -> logw
// 5. Flow + Decoder -> audio waveform
ncnn::Net flow, dec;
// ... z_p -> z -> o (raw audio)