Implementation:Neuml Txtai SQLite Database
| Knowledge Sources | |
|---|---|
| Domains | Database, SQL |
| Last Updated | 2026-02-10 01:00 GMT |
Overview
Concrete tool for SQLite-backed embedded database operations provided by txtai.
Description
SQLite is the default embedded database backend for txtai. It extends Embedded (which extends RDBMS) and provides a lightweight, file-based relational database using Python's built-in sqlite3 module. The class supports optional Write-Ahead Logging (WAL) mode for improved concurrent read performance, enabled via the wal database setting. Connections are created with check_same_thread=False to allow multi-threaded access (with external locking). Custom SQL functions registered via the txtai configuration are installed at the connection level using sqlite3.create_function() with callback tracebacks enabled for debugging. The copy() method provides two strategies for database duplication: if the database has uncommitted changes (in_transaction), it uses iterdump() for a slower but safe copy; otherwise, it uses the efficient SQLite C API backup() method.
Usage
Use SQLite as the database backend for txtai when you want a lightweight, zero-configuration embedded database. This is the default backend and requires no additional package installations. Enable WAL mode for better read concurrency by setting the wal option in the database configuration.
Code Reference
Source Location
- Repository: Neuml_Txtai
- File:
src/python/txtai/database/sqlite.py - Lines: 1-57
Signature
class SQLite(Embedded):
def connect(self, path="")
def getcursor(self)
def rows(self)
def addfunctions(self)
def copy(self, path)
Import
from txtai.database.sqlite import SQLite
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| config | dict | Yes | Database configuration parameters. The wal setting enables WAL journal mode when set to a truthy value. Custom functions are registered from the functions config key.
|
| path | str | No | File system path for the SQLite database file. Defaults to an empty string (in-memory temporary database). |
Outputs
| Name | Type | Description |
|---|---|---|
| connection | sqlite3.Connection | SQLite connection object from connect(), with optional WAL mode enabled.
|
| cursor | sqlite3.Cursor | Standard SQLite cursor from getcursor().
|
| rows | sqlite3.Cursor | The cursor itself acts as the row iterator. |
| new connection | sqlite3.Connection | New connection with data copied via backup() or iterdump() from copy().
|
Usage Examples
from txtai.database.sqlite import SQLite
# Create an SQLite-backed database with WAL mode
config = {
"content": "sqlite",
"sqlite": {"wal": True}
}
db = SQLite(config)
# Initialize the database
db.initialize()
# Insert documents
documents = [
("doc1", {"text": "Introduction to SQLite"}, None),
("doc2", {"text": "Embedded databases for search"}, None),
]
db.insert(documents)
# Save to a file
db.save("/tmp/my_sqlite.db")
# Count stored sections
total = db.count()
# Close connection
db.close()