Environment:Hpcaitech ColossalAI ColossalQA RAG Environment

Knowledge Sources	ColossalAI requirements.txt
Domains	Infrastructure, RAG, NLP
Last Updated	2026-02-09 12:00 GMT

Overview

Python environment for ColossalQA's Retrieval-Augmented Generation pipeline, providing LangChain orchestration, ChromaDB vector storage, sentence-transformers embeddings, and Gradio web UI with pinned dependency versions separate from the main ColossalAI requirements.

Description

This environment defines the dependency stack for the ColossalQA application, a RAG-based question-answering system built on top of ColossalAI. ColossalQA uses LangChain for document loading and chain orchestration, ChromaDB as the vector store backend, sentence-transformers for embedding generation, and Gradio for an interactive web interface. Notably, the PyTorch version requirement (`torch<2.0.0, >=1.12.1`) differs from the main ColossalAI project, reflecting the application's compatibility constraints. The system supports both English and Chinese RAG conversations with dedicated CJK text splitting.

Usage

Use this environment when running ColossalQA RAG pipelines, including document ingestion, vector retrieval, and conversational question answering. Required for the document loader, custom retriever, Chinese text splitter, and the Gradio-based RAG ChatBot web demo.

System Requirements

Category	Requirement	Notes
Python	>= 3.8	Must support all pinned dependency versions
Hardware	NVIDIA GPU recommended	Required for torch and transformer model inference
RAM	8GB+	ChromaDB and embedding models require significant memory
Disk	5GB+	For model weights, vector indexes, and document storage

Dependencies

Core ML Packages

`transformers` >= 4.20.1
`torch` >= 1.12.1, < 2.0.0 (NOTE: differs from main ColossalAI torch requirement)
`sentence-transformers` == 2.2.2
`sentencepiece` == 0.1.99

LangChain Orchestration

`langchain` == 0.0.330
`langchain-experimental` == 0.0.37

Vector Storage

`chromadb` == 0.4.9

LLM API

`openai` == 0.28.0
`tiktoken` == 0.5.1

Document Processing

`unstructured` == 0.10.14
`pypdf` == 3.16.0
`jq` == 1.6.0

Web UI and Networking

`gradio` == 3.44.4
`Requests` == 2.31.0

Model Hub

`modelscope` == 1.9.0

Credentials

`OPENAI_API_KEY`: Required when using OpenAI models for generation or embeddings via the `openai` package.

Quick Install

# Install from the ColossalQA requirements file
cd applications/ColossalQA
pip install -r requirements.txt

# Or install dependencies manually
pip install "transformers>=4.20.1" "torch>=1.12.1,<2.0.0" \
    "langchain==0.0.330" "langchain-experimental==0.0.37" \
    "sentence-transformers==2.2.2" "chromadb==0.4.9" \
    "openai==0.28.0" "tiktoken==0.5.1" \
    "unstructured==0.10.14" "pypdf==3.16.0" "jq==1.6.0" \
    "gradio==3.44.4" "Requests==2.31.0" \
    "sentencepiece==0.1.99" "modelscope==1.9.0"

Code Evidence

ChromaDB vector storage usage from `colossalqa/retriever.py`:

import chromadb

LangChain document loader integration from `colossalqa/data_loader/document_loader.py`:

# Uses langchain document loaders for various file formats
from langchain.document_loaders import ...

CJK text splitting from `colossalqa/text_splitter/chinese_text_splitter.py`:

# Custom text splitter for Chinese/CJK text segmentation

Chinese RAG conversation pipeline from `colossalqa/retrieval_conversation_zh.py`:

# Chinese-language retrieval-augmented conversation using LangChain chains

Gradio web UI from `examples/webui_demo/RAG_ChatBot.py`:

import gradio as gr

Requirements file at `applications/ColossalQA/requirements.txt`:

transformers>=4.20.1
torch<2.0.0, >=1.12.1
langchain==0.0.330
langchain-experimental==0.0.37
sentence-transformers==2.2.2
chromadb==0.4.9
openai==0.28.0
tiktoken==0.5.1
unstructured==0.10.14
pypdf==3.16.0
jq==1.6.0
gradio==3.44.4
Requests==2.31.0
sentencepiece==0.1.99
modelscope==1.9.0

Common Errors

Error Message	Cause	Solution
`torch>=2.0.0 conflicts with torch<2.0.0`	Main ColossalAI installed torch 2.x which conflicts with ColossalQA	Create a separate virtual environment for ColossalQA with `torch<2.0.0`
`ModuleNotFoundError: No module named 'chromadb'`	ChromaDB not installed	`pip install chromadb==0.4.9`
`openai.error.AuthenticationError`	Missing or invalid OpenAI API key	Set `OPENAI_API_KEY` environment variable
`ImportError: langchain` version mismatch	Wrong LangChain version installed	`pip install langchain==0.0.330 langchain-experimental==0.0.37`
`Cannot import sentencepiece`	sentencepiece not installed for tokenizer	`pip install sentencepiece==0.1.99`

Compatibility Notes

PyTorch Version: ColossalQA requires `torch<2.0.0`, which conflicts with the main ColossalAI project. Use a dedicated virtual environment to avoid version conflicts.
LangChain Pinning: Both `langchain` and `langchain-experimental` are pinned to specific versions (0.0.330 and 0.0.37 respectively). LangChain's API changes frequently between releases, so upgrading requires careful testing.
OpenAI SDK: Uses the legacy `openai==0.28.0` API (pre-1.0). This version uses `openai.ChatCompletion.create()` rather than the newer client-based API.
Chinese Language Support: The custom `chinese_text_splitter` and `retrieval_conversation_zh` modules provide CJK-specific text processing. The `sentencepiece` package is required for Chinese tokenization.
Gradio Version: Pinned to `gradio==3.44.4` (Gradio 3.x). Not compatible with Gradio 4.x which introduced breaking API changes.
ModelScope: The `modelscope` package provides access to Chinese model hubs as an alternative to Hugging Face for users in China.

Related Pages

Page Connections

Double-click a node to navigate. Hold to expand connections.

Principle

Implementation

Heuristic

Environment