Principle:Infiniflow Ragflow Document Processing Trigger
| Knowledge Sources | |
|---|---|
| Domains | RAG, Task_Queue, Data_Engineering |
| Last Updated | 2026-02-12 06:00 GMT |
Overview
An asynchronous task dispatch pattern that splits documents into page-range tasks and enqueues them to a Redis-backed worker queue.
Description
Document Processing Trigger initiates the document-to-chunks pipeline by creating Task records in the database and enqueuing them to Redis for consumption by background worker processes. For PDFs, the document is split into page-range tasks based on task_page_size to enable parallel processing. For spreadsheets, splitting is by row ranges. Each task carries a content hash (digest) computed from chunking configuration and page range, enabling incremental re-processing where only changed chunks are recomputed.
Usage
Trigger after documents are uploaded and parser configuration is finalized. This is the entry point to the asynchronous processing pipeline.
Theoretical Basis
The task dispatch follows a fan-out pattern:
- Task splitting: Large documents are divided into page-range tasks for parallel processing across multiple workers
- Content-addressed deduplication: Each task has a digest hash; if the digest matches a previous task, its chunks are reused
- Priority queuing: Tasks can be assigned priorities (0=default) for scheduling control