Workflow:Truera Trulens Snowflake Observability Pipeline
| Knowledge Sources | |
|---|---|
| Domains | LLM_Ops, Evaluation, Enterprise, Snowflake |
| Last Updated | 2026-02-14 08:00 GMT |
Overview
End-to-end process for configuring TruLens to store traces and evaluation results in Snowflake, with support for server-side evaluation and enterprise dashboard deployment.
Description
This workflow covers the enterprise deployment path for TruLens using Snowflake as the backend. It replaces the default SQLite storage with Snowflake event tables, enabling centralized trace storage, server-side feedback evaluation via stored procedures, and dashboard hosting within Snowflake (Streamlit in Snowflake). This path is designed for production environments where traces and evaluations need to be stored alongside other enterprise data in Snowflake.
Usage
Execute this workflow when you need to deploy TruLens in an enterprise environment that uses Snowflake as its data platform. This is appropriate when you need centralized, persistent trace storage accessible by multiple team members, server-side evaluation that does not depend on client compute, integration with existing Snowflake data governance and access controls, or dashboard access through Snowflake's native Streamlit hosting. It replaces the local SQLite or PostgreSQL paths used during development.
Execution Steps
Step 1: Configure Snowflake Connection
Create a SnowflakeConnector with the appropriate credentials and connection parameters. The connector supports direct credential authentication, Snowpark session passthrough, and session creator functions for environments like Snowpark Container Services (SPCS).
Key considerations:
- Provide account, user, password, database, schema, warehouse, and role
- Alternatively pass an existing Snowpark Session or a session creator function
- Set use_account_event_table to control whether traces go to the Snowflake event table
- The database and schema must exist before connecting
Step 2: Initialize TruSession With Snowflake
Pass the SnowflakeConnector to TruSession to route all traces and feedback results to Snowflake instead of the default local database. This makes all subsequent instrumentation and evaluation operations persist to Snowflake.
Key considerations:
- TruSession is a singleton; the connector must be set at initialization
- The connector creates necessary tables and procedures in the specified schema
- Network connectivity to Snowflake is required for all operations
Step 3: Configure Server-Side Evaluation
Enable server-side evaluation by setting init_server_side on the connector. This creates stored procedures and Snowflake tasks that run feedback evaluation within Snowflake compute, eliminating the need for client-side evaluation infrastructure.
What happens:
- Stored procedures are created in the target schema for each feedback function
- A Snowflake task is scheduled to periodically evaluate pending traces
- Evaluation results are written back to the event table
- Client-side compute is not needed for evaluation
Key considerations:
- Server-side evaluation requires appropriate Snowflake permissions (CREATE PROCEDURE, CREATE TASK)
- The Snowflake warehouse must be running for evaluation tasks to execute
- Custom feedback functions may need to be packaged and uploaded as Python UDFs
- Latency is higher than client-side evaluation but scales better
Step 4: Instrument and Record Application
Instrument your LLM application using any TruLens wrapper (TruChain, TruLlama, TruGraph, TruApp, or TruBasicApp) exactly as in the local workflow. The only difference is that traces are exported to the Snowflake event table instead of SQLite.
Key considerations:
- All standard instrumentation patterns work identically
- Traces are batched and exported via the OTEL BatchSpanProcessor
- Network latency to Snowflake may slightly increase trace export time
- The event table stores raw OTEL spans in Snowflake-native format
Step 5: Launch Dashboard
Launch the TruLens dashboard, which can run either locally against the Snowflake backend or within Snowflake as a Streamlit in Snowflake (SiS) application. The SiS mode enables team-wide access through the Snowflake UI without requiring local Python environments.
Key considerations:
- Local dashboard: use run_dashboard(session) as normal; data is read from Snowflake
- Streamlit in Snowflake: set init_sis_dashboard on the connector to create the SiS app
- SPCS mode is available for container-based deployment
- The dashboard queries the event table directly for trace and feedback data
Step 6: Query Results Programmatically
Access evaluation results directly from Snowflake using TruSession methods or standard SQL queries against the event table. This enables integration with existing BI tools, alerting systems, and data pipelines.
Key considerations:
- session.get_leaderboard() works against Snowflake data
- session.get_records_and_feedback() retrieves detailed trace and feedback data
- Standard SQL queries can be run against the event table for custom analysis
- Results can be joined with other Snowflake tables for business context