Implementation:Haifengl Smile InferenceService Setup

Overview

The InferenceService class and InferenceServiceConfig interface together form the model serving bootstrap layer in Smile's serve module. InferenceServiceConfig maps external configuration to a type-safe Java interface via SmallRye Config, while InferenceService is a CDI-managed singleton that eagerly loads all models at application startup and provides model lookup for the REST layer.

This is a Wrapper Doc -- it documents how Smile wraps the Quarkus CDI and SmallRye Config frameworks to implement model lifecycle management.

Source Location

Class	Location
`InferenceService`	serve/src/main/java/smile/serve/InferenceService.java (Lines 43-72)
`InferenceServiceConfig`	serve/src/main/java/smile/serve/InferenceServiceConfig.java (Lines 26-29)
`application.properties`	serve/src/main/resources/application.properties

Import Statements

import smile.serve.InferenceService;
import smile.serve.InferenceServiceConfig;

External Dependencies

Dependency	Annotation/Class	Purpose
Quarkus CDI	`@ApplicationScoped`	Ensures a single instance per application lifecycle
Quarkus CDI	`@Startup`	Triggers eager initialization at boot (not lazy on first injection)
Quarkus CDI	`@Inject`	Constructor injection of the configuration bean
SmallRye Config	`@ConfigMapping(prefix = "smile.serve")`	Maps `smile.serve.*` properties to Java interface methods
JBoss Logging	`org.jboss.logging.Logger`	Logging framework provided by Quarkus

Type

Wrapper Doc (wraps Quarkus CDI and SmallRye Config framework)

API Signatures

InferenceServiceConfig

@ConfigMapping(prefix = "smile.serve")
public interface InferenceServiceConfig {
    /** The location of pre-trained model(s) for inference. */
    String model();
}

This interface uses SmallRye Config's @ConfigMapping to bind the property smile.serve.model to the model() method. Quarkus automatically creates an implementation at build time that reads the value from application.properties, environment variables, or system properties.

InferenceService

@Startup
@ApplicationScoped
public class InferenceService {

    private final Map<String, InferenceModel> models = new TreeMap<>();

    @Inject
    public InferenceService(InferenceServiceConfig config)
    // Loads all .sml models from the configured path at startup

    public List<String> models()
    // Returns list of all registered model IDs

    public InferenceModel getModel(String id) throws NotFoundException
    // Retrieves a model by ID, throws 404 if not found

    public InferenceResponse predict(String model, JsonObject request)
        throws BadRequestException, NotFoundException
    // Delegates prediction to the identified model
}

How Smile Uses Quarkus CDI

Eager Singleton Initialization

The combination of @ApplicationScoped and @Startup is the key pattern:

@Startup            // <-- triggers construction at boot, not on first use
@ApplicationScoped  // <-- single instance for the application's lifetime
public class InferenceService {
    @Inject
    public InferenceService(InferenceServiceConfig config) {
        // Constructor runs at startup, models loaded immediately
    }
}

Without @Startup, Quarkus CDI beans are lazily initialized (created on first injection). For a model server, lazy initialization would cause the first HTTP request to block while models are deserialized -- potentially for seconds. The @Startup annotation ensures models are loaded before the server accepts any requests.

Constructor Injection

The @Inject annotation on the constructor tells Quarkus CDI to resolve and inject the InferenceServiceConfig bean automatically. Since InferenceServiceConfig is a @ConfigMapping interface, Quarkus generates its implementation and provides it to the constructor.

How Smile Uses SmallRye Config

Configuration Mapping

SmallRye Config's @ConfigMapping provides type-safe access to configuration properties:

@ConfigMapping(prefix = "smile.serve")
public interface InferenceServiceConfig {
    String model();   // maps to property: smile.serve.model
}

The prefix element specifies the property namespace. Each method in the interface maps to a property key formed by combining the prefix with the method name: smile.serve + . + model = smile.serve.model.

Configuration Sources

Quarkus resolves smile.serve.model from multiple sources (in priority order):

System properties: -Dsmile.serve.model=/path/to/models
Environment variables: SMILE_SERVE_MODEL=/path/to/models
application.properties file: smile.serve.model=../model

Configuration Examples

application.properties

# Point to a directory of .sml files (loads all models)
smile.serve.model=../model

# Or point to a single .sml file
# smile.serve.model=/opt/models/iris-classifier.sml

# Quarkus REST path prefix
quarkus.rest.path=/api/v1

# Development profile overrides
%dev.quarkus.http.port=8888
%test.smile.serve.model=model

Environment Variable Override

# Override model path at runtime without changing application.properties
export SMILE_SERVE_MODEL=/opt/production/models
java -jar smile-serve-runner.jar

Model Loading Logic

The constructor implements a two-mode loading strategy:

@Inject
public InferenceService(InferenceServiceConfig config) {
    var path = Paths.get(config.model()).toAbsolutePath().normalize();
    if (Files.isRegularFile(path)) {
        // Mode 1: Single file -- load one model
        loadModel(path);
    } else if (Files.isDirectory(path)) {
        // Mode 2: Directory -- load all .sml files
        try (Stream<Path> files = Files.list(path)) {
            files.forEach(file -> {
                if (Files.isRegularFile(file) && file.toString().endsWith(".sml")) {
                    loadModel(file);
                }
            });
        } catch (IOException ex) {
            logger.error(ex);
        }
    } else {
        logger.errorf("'%s' is not a regular file", path);
    }
}

Each model is deserialized via Read.object(), validated as a Model instance, wrapped in an InferenceModel, and stored in the TreeMap keyed by model ID:

private void loadModel(Path path) {
    try {
        logger.infof("Loading model from '%s'", path);
        var obj = Read.object(path);
        if (obj instanceof Model dummy) {
            var model = new InferenceModel(dummy, path);
            models.put(model.id(), model);
        } else {
            logger.errorf("'%s' is not a valid model", path);
        }
    } catch (Exception ex) {
        logger.errorf(ex, "Failed to load model '%s'", path);
    }
}

The model ID is derived from the model's id and version tags (or defaults to the filename and version "1"):

this.id = model.getTag(Model.ID, Paths.getFileName(path)) + "-"
        + model.getTag(Model.VERSION, "1");

Model Lookup API

// List all model IDs
public List<String> models() {
    return new ArrayList<>(models.keySet());
}

// Get a specific model (throws NotFoundException / HTTP 404 if not found)
public InferenceModel getModel(String id) throws NotFoundException {
    var model = models.get(id);
    if (model == null) throw new NotFoundException(id);
    return model;
}

The NotFoundException is from jakarta.ws.rs, which Quarkus automatically maps to an HTTP 404 response when thrown from a JAX-RS resource.

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment