Implementation:Tensorflow Serving Resource Util

Knowledge Sources	Tensorflow_Serving
Domains	Resource Management, Arithmetic
Last Updated	2026-02-13 00:00 GMT

Overview

A utility class that provides arithmetic, comparison, and normalization operations on resource allocations represented as protobuf messages, supporting both bound (device-instance-specific) and unbound resource entries.

Description

ResourceUtil performs algebra on ResourceAllocation protobuf messages, which represent quantities of resources (e.g., RAM, processing cores) across devices (e.g., CPU, GPU) with optional binding to specific device instances. The class is constructed with a device configuration map (device name to instance count) and provides operations including: Add (sum two allocations), Subtract (difference with non-negativity check), Multiply (scalar multiplication), Equal (equality modulo normalization), LessThanOrEqual (comparison supporting unbound-to-bound matching), Max/Min (element-wise max/min), Overbind (convert unbound to bound by replicating to all instances), and Normalize (remove zero entries and bind single-instance devices). All operations work in a normalized form where zero-quantity entries are removed and single-instance devices are automatically bound. The VerifyValidity method checks that allocations only reference valid devices and instances without duplicate entries. Bound and unbound entries are kept separate in arithmetic operations to preserve the distinction between "allocated to a specific device instance" and "allocated to some unspecified instance."

Usage

Use this to perform resource bookkeeping in the serving system, such as determining whether there are enough resources to load a new servable, computing the total resources used by loaded models, or checking resource constraints during model management decisions.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/resources/resource_util.h (header), tensorflow_serving/resources/resource_util.cc (implementation)
• Lines: 1-251 (header), 1-586 (implementation)

Signature

class ResourceUtil {
 public:
  struct Options {
    std::map<string, uint32> devices;
  };
  explicit ResourceUtil(const Options& options);

  virtual Status VerifyValidity(const ResourceAllocation& allocation) const;
  virtual ResourceAllocation Normalize(const ResourceAllocation& allocation) const;
  virtual bool IsNormalized(const ResourceAllocation& allocation) const;
  bool IsBound(const ResourceAllocation& allocation) const;

  Resource CreateBoundResource(const string& device, const string& kind,
                               uint32 device_instance = 0) const;
  uint64_t GetQuantity(const Resource& resource,
                       const ResourceAllocation& allocation) const;
  void SetQuantity(const Resource& resource, uint64_t quantity,
                   ResourceAllocation* allocation) const;

  void Add(const ResourceAllocation& to_add, ResourceAllocation* base) const;
  bool Subtract(const ResourceAllocation& to_subtract, ResourceAllocation* base) const;
  void Multiply(uint64_t multiplier, ResourceAllocation* base) const;
  bool Equal(const ResourceAllocation& lhs, const ResourceAllocation& rhs) const;
  bool LessThanOrEqual(const ResourceAllocation& lhs, const ResourceAllocation& rhs) const;
  ResourceAllocation Overbind(const ResourceAllocation& allocation) const;
  ResourceAllocation Max(const ResourceAllocation& lhs, const ResourceAllocation& rhs) const;
  ResourceAllocation Min(const ResourceAllocation& lhs, const ResourceAllocation& rhs) const;
};

Import

#include "tensorflow_serving/resources/resource_util.h"

I/O Contract

Inputs

Name	Type	Required	Description
options	Options	Yes (constructor)	Device configuration mapping device names to instance counts
allocation	ResourceAllocation	Yes	Resource allocation protobuf to operate on
resource	Resource	Yes (Get/SetQuantity)	A specific resource (device + kind + optional instance)

Outputs

Name	Type	Description
Status	tensorflow::Status	OK if the allocation is valid, error otherwise
ResourceAllocation	ResourceAllocation	Resulting allocation from Normalize, Overbind, Max, Min
bool	bool	Result of comparison or subtraction feasibility

Usage Examples

Resource Arithmetic

ResourceUtil::Options options;
options.devices = {{"CPU", 1}, {"GPU", 2}};
ResourceUtil util(options);

ResourceAllocation total;
// ... populate total with available resources ...

ResourceAllocation model_resources;
// ... populate model_resources with a model's requirements ...

// Check if model fits
ResourceAllocation proposed = util.Overbind(model_resources);
util.Add(used_resources, &proposed);
if (util.LessThanOrEqual(proposed, total)) {
  // Model fits, proceed with loading
  util.Add(model_resources, &used_resources);
}

Related Pages

• Principle:Tensorflow_Serving_Resource_Arithmetic