Principle:Tensorflow Serving JSON Request Formatting

Knowledge Sources	TF Serving REST API
Domains	Data_Serialization, API_Design
Last Updated	2026-02-13 17:00 GMT

Overview

A data conversion process that transforms JSON request payloads into TensorFlow TensorProto messages for model inference.

Description

JSON request formatting bridges the gap between human-readable JSON and TensorFlow's internal tensor representation. TensorFlow Serving supports two JSON formats:

• Row format ({"instances": [...]}): Each element is one input example. Intuitive for single-input models.
• Columnar format ({"inputs": {...}}): Each key maps to a named tensor with all examples. Efficient for multi-input models.

The conversion process parses the JSON using RapidJSON, determines tensor types from the model's SignatureDef metadata, and fills TensorProto messages with correctly typed values. Supported types include DT_FLOAT, DT_DOUBLE, DT_INT32, DT_INT64, DT_STRING, DT_BOOL, DT_UINT32, DT_UINT64.

Usage

Use row format for simple single-input models and columnar format for complex multi-input models or when sending large batches efficiently.

Theoretical Basis

# Row format: each element = one input instance
{"instances": [
    [1.0, 2.0, 3.0],  # instance 1
    [4.0, 5.0, 6.0]   # instance 2
]}

# Columnar format: each key = one named tensor
{"inputs": {
    "feature_a": [[1.0, 2.0], [3.0, 4.0]],
    "feature_b": [[5.0], [6.0]]
}}

# Both produce the same PredictRequest with 2 input examples

Related Pages

Implemented By

• Implementation:Tensorflow_Serving_FillPredictRequestFromJson