Implementation:Tensorflow Serving Resnet Client GRPC Inference

Knowledge Sources	TensorFlow Serving
Domains	Inference, Kubernetes
Last Updated	2026-02-13 17:00 GMT

Overview

Concrete tool for querying a Kubernetes-deployed TensorFlow Serving ResNet model via gRPC, provided by the example client scripts.

Description

resnet_client_grpc.py is a Python gRPC client that:

Downloads a test image (cat.jpg from tensorflow.org) or loads a local image via --image
Preprocesses the image into a tensor using tf.make_tensor_proto()
Creates a gRPC channel to the server (--server flag, default "localhost:8500")
Builds a PredictRequest with model_spec.name = "resnet" and signature_name = "serving_default"
Sends the request via PredictionServiceStub(channel).Predict() with 10-second timeout
Prints the predicted ImageNet class (argmax of 1001-element output vector)

Usage

Run after deploying the Kubernetes resources and obtaining the LoadBalancer external IP. Pass the external IP as the --server flag.

Code Reference

Source Location

Repository: tensorflow/serving
File: tensorflow_serving/example/resnet_client_grpc.py (L46-74)

Signature

def main():
    """
    Send gRPC Predict request to TF Serving.

    CLI flags:
      --server: str (default "localhost:8500") - gRPC server address
      --image: str (default "") - optional local image path
    """

Import

import grpc
import numpy as np
import requests
import tensorflow as tf
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc

I/O Contract

Inputs

Name	Type	Required	Description
--server	str	No	gRPC server address (default "localhost:8500")
--image	str	No	Local image path (downloads cat.jpg if not set)

Outputs

Name	Type	Description
Prediction class	int	Predicted ImageNet class index (0-1000)
response.outputs	TensorProto	1001-element float vector of class probabilities

Usage Examples

Query Kubernetes Deployment

# Get external IP
EXTERNAL_IP=$(kubectl get svc resnet-service -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# Query with default cat image
python tensorflow_serving/example/resnet_client_grpc.py \
    --server=${EXTERNAL_IP}:8500

# Query with custom image
python tensorflow_serving/example/resnet_client_grpc.py \
    --server=${EXTERNAL_IP}:8500 \
    --image=/path/to/my_image.jpg

Programmatic Usage

import grpc
import numpy as np
import tensorflow as tf
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc

# 1. Connect to server
channel = grpc.insecure_channel('EXTERNAL_IP:8500')
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

# 2. Build request
request = predict_pb2.PredictRequest()
request.model_spec.name = 'resnet'
request.model_spec.signature_name = 'serving_default'

# 3. Prepare image data (224x224x3 float32)
image_data = np.random.rand(1, 224, 224, 3).astype(np.float32)
request.inputs['input_1'].CopyFrom(
    tf.make_tensor_proto(image_data)
)

# 4. Send request
response = stub.Predict(request, 10.0)  # 10 second timeout

# 5. Parse response
output = np.array(response.outputs['activation_49'].float_val)
predicted_class = np.argmax(output)
print(f'Prediction class: {predicted_class}')

Related Pages

Implements Principle

Principle:Tensorflow_Serving_Kubernetes_Service_Query

Requires Environment

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment