Implementation:Tensorflow Serving simple loader h

Knowledge Sources	Tensorflow_Serving
Domains	Model Serving, Core Framework
Last Updated	2026-02-13 00:00 GMT

Overview

SimpleLoader is a template Loader wrapper that simplifies servable loading using a creator callback and resource estimator, and SimpleLoaderSourceAdapter bundles a SourceAdapter with SimpleLoader creation.

Description

SimpleLoader<ServableType> is a convenience wrapper around the Loader interface for common use cases where:

• The servable's estimated resource footprint is static.
• The servable can be loaded by invoking a no-argument closure (or a closure with metadata).
• The servable can be unloaded by invoking its destructor.

Users provide a Creator callback (invoked during Load()) and a ResourceEstimator callback. The resource estimate is memoized for efficiency. On Unload(), if a main-memory footprint estimate is available, that amount of memory is released to the OS via MallocExtension_ReleaseToSystem().

SimpleLoader supports two resource estimators: one for during-load and one for post-load, enabling transient memory to be released after loading.

SimpleLoaderSourceAdapter<DataType, ServableType> bundles together the SourceAdapter and Loader concepts. It translates aspired-version items one at a time (like UnarySourceAdapter) and creates SimpleLoader instances for each item. Subclasses provide a Creator that takes a DataType and produces a ServableType.

Both classes provide EstimateNoResources() for test or best-effort environments that do not track resource usage.

Usage

Use SimpleLoader directly when you have a simple factory function that creates a servable. Use SimpleLoaderSourceAdapter when building a SourceAdapter pipeline that creates loaders from data objects (e.g., storage paths). This is the most commonly used Loader implementation in TensorFlow Serving.

Code Reference

Source Location

• Repository: Tensorflow_Serving
• File: tensorflow_serving/core/simple_loader.h
• Lines: 1-417

Signature

template <typename ServableType>
class SimpleLoader : public Loader {
 public:
  using Creator = std::function<Status(std::unique_ptr<ServableType>*)>;
  using CreatorWithMetadata =
      std::function<Status(const Metadata&, std::unique_ptr<ServableType>*)>;
  using ResourceEstimator = std::function<Status(ResourceAllocation*)>;

  static ResourceEstimator EstimateNoResources();

  SimpleLoader(Creator creator, ResourceEstimator resource_estimator);
  SimpleLoader(CreatorWithMetadata creator_with_metadata,
               ResourceEstimator resource_estimator);
  SimpleLoader(Creator creator, ResourceEstimator resource_estimator,
               ResourceEstimator post_load_resource_estimator);

  Status EstimateResources(ResourceAllocation* estimate) const override;
  Status Load() override;
  Status LoadWithMetadata(const Metadata& metadata) override;
  void Unload() override;
  AnyPtr servable() override;
};

template <typename DataType, typename ServableType>
class SimpleLoaderSourceAdapter
    : public UnarySourceAdapter<DataType, std::unique_ptr<Loader>> {
 public:
  using Creator = std::function<Status(const DataType&, std::unique_ptr<ServableType>*)>;
  using ResourceEstimator = std::function<Status(const DataType&, ResourceAllocation*)>;
  static ResourceEstimator EstimateNoResources();

 protected:
  SimpleLoaderSourceAdapter(Creator creator, ResourceEstimator resource_estimator);
  Status Convert(const DataType& data, std::unique_ptr<Loader>* loader) final;
};

Import

#include "tensorflow_serving/core/simple_loader.h"

I/O Contract

Inputs

Name	Type	Required	Description
creator	Creator or CreatorWithMetadata	Yes	Callback invoked during Load() to create the servable
resource_estimator	ResourceEstimator	Yes	Callback to estimate the servable's resource footprint
post_load_resource_estimator	ResourceEstimator	No	Optional callback for post-load resource estimation (enables transient memory release)

Outputs

Name	Type	Description
Load()	Status	Invokes the creator to construct the servable; returns error on failure
Unload()	void	Destroys the servable and releases estimated memory to the OS
servable()	AnyPtr	Returns a type-erased pointer to the loaded servable object
EstimateResources()	Status	Fills in a ResourceAllocation with the memoized resource estimate

Usage Examples

Creating a SimpleLoader

#include "tensorflow_serving/core/simple_loader.h"

using namespace tensorflow::serving;

auto creator = [](std::unique_ptr<time_t>* servable) {
  servable->reset(new time_t);
  **servable = time(nullptr);
  return Status();
};

auto resource_estimator = [](ResourceAllocation* estimate) {
  estimate->Clear();
  return Status();
};

std::unique_ptr<Loader> loader(
    new SimpleLoader<time_t>(creator, resource_estimator));

Using SimpleLoaderSourceAdapter

#include "tensorflow_serving/core/simple_loader.h"

using namespace tensorflow::serving;

class MyAdapter : public SimpleLoaderSourceAdapter<StoragePath, MyServable> {
 public:
  MyAdapter()
      : SimpleLoaderSourceAdapter(
            [](const StoragePath& path, std::unique_ptr<MyServable>* s) {
              s->reset(new MyServable(path));
              return Status();
            },
            EstimateNoResources()) {}
  ~MyAdapter() override { Detach(); }
};

Related Pages

• Principle:Tensorflow_Serving_Loader_Abstraction