Implementation:Tensorflow Serving Tfrt Multi Inference Test
| Knowledge Sources | |
|---|---|
| Domains | Testing, Inference, TFRT |
| Last Updated | 2026-02-13 00:00 GMT |
Overview
Test suite validating the TFRT-based multi-inference functionality which executes multiple inference tasks using the TFRT runtime.
Description
This test file validates the TFRT multi-inference implementation that runs multiple classification and regression tasks using the TFRT execution engine. The TfrtMultiInferenceTest fixture initializes a TFRT runtime with 4 inter-op threads, sets up a ServerCore with TfrtSavedModelSourceAdapterConfig, and obtains ServableHandle<Servable> for executing multi-inference. It tests both the Servable::MultiInference interface and the direct RunMultiInference function with TFRT SavedModel.
Key areas tested include:
- Missing input validation
- Undefined and duplicate signature detection
- Inconsistent model specs across tasks
- Unsupported signature types
- Valid single and multiple regression signatures
- Mixed regression and classification in one request
Usage
Run these tests to validate changes to the TFRT multi-inference pipeline or the TFRT servable multi-inference interface.
Code Reference
Source Location
- Repository: Tensorflow_Serving
- File: tensorflow_serving/servables/tensorflow/tfrt_multi_inference_test.cc
- Lines: 1-305
Test Fixture
class TfrtMultiInferenceTest : public ::testing::Test {
public:
static void SetUpTestSuite() {
tfrt_stub::SetGlobalRuntime(
tfrt_stub::Runtime::Create(/*num_inter_op_threads=*/4));
CreateServerCore(&server_core_);
}
protected:
ServerCore* GetServerCore() { return server_core_.get(); }
absl::Status GetServableHandle(ServableHandle<Servable>* servable);
const int64_t servable_version_ = kTestModelVersion;
private:
static std::unique_ptr<ServerCore> server_core_;
};
Build Target
bazel test //tensorflow_serving/servables/tensorflow:tfrt_multi_inference_test
Test Coverage
Key Test Cases
| Test Name | Category | Description |
|---|---|---|
| MissingInputTest | Validation | Tests error when input is empty |
| UndefinedSignatureTest | Validation | Tests error on non-existent signature name |
| InconsistentModelSpecsInRequestTest | Validation | Tests error when tasks reference different models |
| EvaluateDuplicateFunctionsTest | Validation | Tests error on duplicate function signatures |
| UsupportedSignatureTypeTest | Validation | Tests error on unsupported method types |
| ValidSingleSignatureTest | Integration | Tests successful single regression signature |
| MultipleValidRegressSignaturesTest | Integration | Tests multiple regression signatures in one request |
| RegressAndClassifySignaturesTest | Integration | Tests mixed regression and classification |
Usage Examples
Test Pattern
TEST_F(TfrtMultiInferenceTest, ValidSingleSignatureTest) {
MultiInferenceRequest request;
AddInput({{"x", 2}}, &request);
PopulateTask("regress_x_to_y", kRegressMethodName, -1, request.add_tasks());
MultiInferenceResponse response;
ServableHandle<Servable> servable;
TF_ASSERT_OK(GetServableHandle(&servable));
TF_ASSERT_OK(servable->MultiInference({}, request, &response));
// Validate regression result: 0.5 * 2 + 2 = 3
}