Workflow:Hpcaitech ColossalAI RAG Application
| Knowledge Sources | |
|---|---|
| Domains | LLMs, RAG, Information_Retrieval |
| Last Updated | 2026-02-09 03:00 GMT |
Overview
End-to-end process for building a Retrieval-Augmented Generation (RAG) application with multilingual support using ColossalQA's document retrieval and conversational QA framework.
Description
This workflow implements a RAG pipeline that combines document retrieval with language model generation to answer questions based on custom knowledge bases. It uses LangChain as the underlying framework with Chroma vector stores for document retrieval and supports multiple LLM backends (local HuggingFace models, ColossalCloud API, PanGu, and ChatGPT). The system includes bilingual support for Chinese and English with language-specific text splitting, conversation memory management, and an optional Streamlit web UI for interactive use.
Usage
Execute this workflow when you need to build a question-answering system that can answer questions based on a specific document corpus (company knowledge base, product documentation, research papers). This is suitable when the language model's parametric knowledge is insufficient and you need grounded answers from specific documents.
Execution Steps
Step 1: Document Loading
Load source documents from various formats (text files, markdown, JSON, PDFs) using the DocumentLoader utility. Documents are tagged with metadata including source name and file path for attribution.
Key considerations:
- Supports multiple file formats via LangChain document loaders
- Each document source is identified by a name for retrieval attribution
- Table data can be loaded separately using the TableDataLoader for structured data
Step 2: Text Splitting and Chunking
Split loaded documents into overlapping chunks suitable for embedding and retrieval. Use language-appropriate splitters for optimal chunk quality.
What happens:
- For Chinese: use ChineseTextSplitter with sentence-level splitting
- For English: use RecursiveCharacterTextSplitter with standard chunk sizes
- Configure chunk_size and chunk_overlap for the target use case
- Preserve metadata through the splitting process
Step 3: Embedding and Indexing
Generate vector embeddings for document chunks and store them in a Chroma vector database. The CustomRetriever manages multiple vector stores (one per source or merged) with SQLite-based record management for incremental updates.
What happens:
- Initialize embedding model (default: moka-ai/m3e-base for multilingual)
- Create Chroma vector store per document source or as a single merged store
- Create SQL record manager for tracking indexed documents
- Index documents with support for incremental updates or full rebuild
- Store embeddings in persistent Chroma collections
Step 4: Retrieval Chain Configuration
Set up the retrieval QA chain that combines document retrieval with language model generation. Configure the conversation template, memory management, and LLM backend.
What happens:
- Configure CustomRetriever with top-k setting for retrieval
- Initialize LLM backend (local model, ColossalCloud, or ChatGPT)
- Create RetrievalConversation chain with language-specific prompts
- Configure conversation memory for multi-turn dialogue context
- Set up summarization memory for long conversations
Step 5: Query Processing and Response Generation
Process user queries through the RAG pipeline: detect language, retrieve relevant documents, and generate grounded responses using the language model.
What happens:
- Detect query language using naive language detection
- Route to appropriate language-specific retriever and chain
- Retrieve top-k relevant document chunks via similarity search
- Construct prompt with retrieved context and conversation history
- Generate response using the configured LLM
- Update conversation memory with the interaction
Step 6: Deployment (Optional)
Deploy the RAG application as a web service using the Streamlit web UI or the FastAPI HTTP server for production use.
Deployment options:
- Streamlit web UI: Interactive chat interface with document upload
- FastAPI server: HTTP API for integration with external systems
- Command-line interface: Direct terminal interaction for testing