Implementation:ArroyoSystems Arroyo Udf Common Types

Knowledge Sources	ArroyoSystems_Arroyo
Domains	Streaming, UDF, FFI
Last Updated	2026-02-08 08:00 GMT

Overview

The arroyo-udf-common crate provides shared FFI-safe types for passing Arrow array data between the Arroyo engine and dynamically loaded UDF dylibs, including FfiArraySchema, FfiArrays, RunResult, and ArrowDatum.

Description

This crate defines the core data types used at the FFI boundary between the Arroyo runtime and user-defined functions compiled as dynamic libraries. All key types use #[repr(C)] for C-compatible memory layout:

FfiArraySchema -- A pair of FFI_ArrowArray and FFI_ArrowSchema representing a single Arrow array with its schema. Constructed from ArrayData via from_data using Arrow's to_ffi function.

FfiArrays -- A C-compatible vector of FfiArraySchema values, representing the argument list passed to a UDF. Uses raw pointer management (leak on creation, from_raw_parts on consumption) to safely transfer ownership across the FFI boundary. The error flag indicates UDF execution failure.

RunResult -- An enum wrapping either Ok(FfiArraySchema) (successful UDF result) or Err (execution failure).

ArrowDatum -- A Rust enum representing a single scalar value for any supported Arrow type (Bool, U32, U64, I32, I64, F32, F64, String, Bytes, Timestamp). The append_to method appends the value to an appropriate ArrayBuilder by downcasting at runtime, enabling row-by-row result construction for async UDFs.

Usage

Used by both the Arroyo engine (to prepare arguments for UDF calls) and by the UDF dylib runtime (to return results). This crate is a dependency of arroyo-udf-plugin and is linked into user-compiled UDF shared libraries.

Code Reference

Source Location

Repository: ArroyoSystems_Arroyo
File: crates/arroyo-udf/arroyo-udf-common/src/lib.rs

Signature

#[repr(C)]
#[derive(Debug)]
pub struct FfiArraySchema(pub FFI_ArrowArray, pub FFI_ArrowSchema);

impl FfiArraySchema {
    pub fn from_data(data: ArrayData) -> Self;
}

#[repr(C)]
pub struct FfiArrays {
    ptr: *mut FfiArraySchema,
    len: usize,
    capacity: usize,
    error: bool,
}

impl FfiArrays {
    pub fn from_vec(value: Vec<ArrayData>) -> Self;
    pub fn into_vec(self) -> Vec<ArrayData>;
}

#[repr(C)]
pub enum RunResult {
    Ok(FfiArraySchema),
    Err,
}

pub enum ArrowDatum {
    Bool(Option<bool>),
    U32(Option<u32>),
    U64(Option<u64>),
    I32(Option<i32>),
    I64(Option<i64>),
    F32(Option<f32>),
    F64(Option<f64>),
    String(Option<String>),
    Bytes(Option<Vec<u8>>),
    Timestamp(Option<SystemTime>),
}

impl ArrowDatum {
    pub fn append_to(self, builder: &mut dyn ArrayBuilder);
}

Import

use arroyo_udf_common::{FfiArraySchema, FfiArrays, RunResult, ArrowDatum};

I/O Contract

Inputs

Name	Type	Required	Description
array_data	Vec<ArrayData>	Yes	Arrow array data to be converted to FFI-safe FfiArrays for UDF invocation
datum_value	ArrowDatum variant	Yes	Scalar value to be appended to an ArrayBuilder for result construction

Outputs

Name	Type	Description
ffi_arrays	FfiArrays	C-compatible array vector ready for passing across the FFI boundary
array_data	Vec<ArrayData>	Reconstituted Arrow arrays from FfiArrays.into_vec()
run_result	RunResult	UDF execution result containing either the output array or an error flag

Usage Examples

use arroyo_udf_common::{FfiArrays, ArrowDatum};
use arrow::array::{ArrayData, Int64Builder, ArrayBuilder};

// Prepare arguments for a UDF call
let args: Vec<ArrayData> = vec![input_array.to_data()];
let ffi_args = FfiArrays::from_vec(args);

// Consume results from a UDF call
let result_arrays = ffi_args.into_vec();

// Append a scalar result to a builder (used in async UDF)
let datum = ArrowDatum::I64(Some(42));
let mut builder = Int64Builder::new();
datum.append_to(&mut builder);

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment