Implementation:Apache Paimon FunctionDefinition

Knowledge Sources	Apache_Paimon
Domains	Functions, User Defined Functions
Last Updated	2026-02-08 00:00 GMT

Overview

FunctionDefinition is a polymorphic interface defining different types of user-defined function implementations in Apache Paimon.

Description

The FunctionDefinition interface and its implementations provide a flexible type system for representing different kinds of function definitions in Paimon. Using Jackson's polymorphic deserialization, the interface supports three distinct function types: file-based functions (compiled code in JARs), SQL functions (inline SQL expressions), and lambda functions (language-specific inline code).

FileFunctionDefinition represents traditional UDFs implemented in languages like Java or Scala, packaged as JAR files. It includes file resources, the programming language, class name, and function name. This type is suitable for complex functions requiring multiple dependencies or performance-critical implementations.

SQLFunctionDefinition represents functions defined as SQL expressions, enabling simple transformations and calculations to be defined inline without external compilation. This type integrates naturally with SQL-based execution engines.

LambdaFunctionDefinition supports inline function definitions in various languages (like Python or JavaScript), combining the flexibility of scripting languages with structured function metadata. Each lambda includes both the code definition and the language identifier.

The FunctionFileResource nested class represents external file dependencies for file-based functions, specifying resource types (JAR, file, etc.) and URIs for loading.

Usage

Use FunctionDefinition when creating or registering user-defined functions in the catalog. Choose the appropriate subtype based on how the function is implemented: FileFunctionDefinition for compiled code, SQLFunctionDefinition for SQL expressions, or LambdaFunctionDefinition for inline scripts. The factory methods provide convenient construction of each type.

Code Reference

Source Location

• Repository: Apache_Paimon
• File: paimon-api/src/main/java/org/apache/paimon/function/FunctionDefinition.java

Signature

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "type")
@JsonSubTypes({
    @JsonSubTypes.Type(value = FileFunctionDefinition.class, name = "file"),
    @JsonSubTypes.Type(value = SQLFunctionDefinition.class, name = "sql"),
    @JsonSubTypes.Type(value = LambdaFunctionDefinition.class, name = "lambda")
})
public interface FunctionDefinition {
    static FunctionDefinition file(
        List<FunctionFileResource> fileResources,
        String language,
        String className,
        String functionName)

    static FunctionDefinition sql(String definition)

    static FunctionDefinition lambda(String definition, String language)

    // Nested classes
    final class FileFunctionDefinition implements FunctionDefinition
    final class SQLFunctionDefinition implements FunctionDefinition
    final class LambdaFunctionDefinition implements FunctionDefinition
    class FunctionFileResource
}

Import

import org.apache.paimon.function.FunctionDefinition;

I/O Contract

Inputs

Name	Type	Required	Description
fileResources	List<FunctionFileResource>	Yes	File resources for file-based functions
language	String	Yes	Programming language (e.g., "java", "scala", "python")
className	String	Yes	Fully qualified class name for file-based functions
functionName	String	Yes	Function name within the class
definition	String	Yes	SQL expression or lambda code definition
resourceType	String	Yes	Type of resource (e.g., "jar", "file")
uri	String	Yes	URI to the resource file

Outputs

Name	Type	Description
functionDefinition	FunctionDefinition	Polymorphic function definition instance
fileResources	List<FunctionFileResource>	File dependencies (for FileFunctionDefinition)
language	String	Programming language identifier
className	String	Class name (for FileFunctionDefinition)
functionName	String	Function name (for FileFunctionDefinition)
definition	String	SQL or lambda code definition

Usage Examples

// Creating a file-based function definition (Java UDF)
List<FunctionFileResource> resources = Arrays.asList(
    new FunctionFileResource("jar", "hdfs:///udfs/my-functions.jar")
);
FunctionDefinition fileDef = FunctionDefinition.file(
    resources,
    "java",
    "com.example.MyFunction",
    "evaluate"
);

// Creating a SQL function definition
FunctionDefinition sqlDef = FunctionDefinition.sql(
    "CASE WHEN amount > 100 THEN 'high' ELSE 'low' END"
);

// Creating a lambda function definition (Python)
FunctionDefinition pythonLambda = FunctionDefinition.lambda(
    "lambda x: x * 2 + 1",
    "python"
);

// Creating a lambda function definition (JavaScript)
FunctionDefinition jsLambda = FunctionDefinition.lambda(
    "function(x) { return x.toUpperCase(); }",
    "javascript"
);

// Using in function registration
Map<String, FunctionDefinition> definitions = new HashMap<>();
definitions.put("spark", fileDef);
definitions.put("flink", sqlDef);
definitions.put("python", pythonLambda);

Function func = new FunctionImpl(
    identifier,
    inputParams,
    outputParams,
    true,
    definitions,
    "Multi-engine function",
    options
);

// Accessing file function details
if (funcDef instanceof FunctionDefinition.FileFunctionDefinition) {
    FileFunctionDefinition fileDef = (FileFunctionDefinition) funcDef;
    List<FunctionFileResource> jars = fileDef.fileResources();
    String className = fileDef.className();
    String language = fileDef.language();
}

// Accessing SQL function details
if (funcDef instanceof FunctionDefinition.SQLFunctionDefinition) {
    SQLFunctionDefinition sqlDef = (SQLFunctionDefinition) funcDef;
    String sqlCode = sqlDef.definition();
}

// Accessing lambda function details
if (funcDef instanceof FunctionDefinition.LambdaFunctionDefinition) {
    LambdaFunctionDefinition lambdaDef = (LambdaFunctionDefinition) funcDef;
    String code = lambdaDef.definition();
    String lang = lambdaDef.language();
}

// Complex file-based function with multiple dependencies
List<FunctionFileResource> complexResources = Arrays.asList(
    new FunctionFileResource("jar", "s3://bucket/main-udf.jar"),
    new FunctionFileResource("jar", "s3://bucket/dependencies.jar"),
    new FunctionFileResource("file", "s3://bucket/config.properties")
);
FunctionDefinition complexDef = FunctionDefinition.file(
    complexResources,
    "scala",
    "org.example.ComplexUDF",
    "process"
);

Related Pages

• Principle:Apache_Paimon_Catalog_Object_Model