Heuristic:Neuml Txtai MacOS Stability Workarounds
| Knowledge Sources | |
|---|---|
| Domains | Debugging, Infrastructure |
| Last Updated | 2026-02-10 00:00 GMT |
Overview
Collection of environment variable workarounds required to prevent segmentation faults and crashes on macOS and Windows.
Description
txtai on macOS (and Windows) can encounter segmentation faults caused by Faiss OpenMP threading conflicts, MPS device incompatibilities, and llama.cpp Metal acceleration issues. The codebase includes automatic workarounds (setting `OMP_NUM_THREADS` and `KMP_DUPLICATE_LIB_OK` at import time for Faiss) and documents manual workarounds for MPS and Metal. These issues stem from platform-specific library linking conflicts, not txtai bugs.
Usage
Apply these workarounds when encountering segmentation faults, OMP errors, or MPS-related crashes on macOS or Windows. The Faiss workarounds are applied automatically at import time. The MPS and Metal workarounds require manual environment variable configuration.
The Insight (Rule of Thumb)
- Action: Set the following environment variables on macOS:
- `export OMP_NUM_THREADS=1` — Disables OpenMP multithreading to prevent Faiss segfaults
- `export KMP_DUPLICATE_LIB_OK=TRUE` — Works around OMP Error #15 (duplicate library loading)
- `export PYTORCH_MPS_DISABLE=1` — Disables PyTorch MPS device (Apple Silicon) to prevent unsupported operation errors
- `export LLAMA_NO_METAL=1` — Disables llama.cpp Metal acceleration on macOS
- Value: The first two are set automatically by txtai when the Faiss module loads on macOS/Windows. The last two must be set manually if issues occur.
- Trade-off: Disabling MPS forces CPU inference on Apple Silicon, which is slower but stable. Disabling Metal disables GPU offloading for llama.cpp GGUF models. Setting `OMP_NUM_THREADS=1` reduces Faiss parallelism but prevents crashes.
Reasoning
Faiss uses OpenMP for parallel similarity search. On macOS, the system ships with Apple's clang which links a different OpenMP implementation than what Faiss expects, causing segmentation faults. Setting `OMP_NUM_THREADS=1` eliminates the threading conflict. The `KMP_DUPLICATE_LIB_OK` workaround addresses a specific Intel OpenMP error that occurs when multiple copies of the library are loaded (common in conda environments).
MPS (Metal Performance Shaders) on Apple Silicon is still maturing and some PyTorch operations used by transformers models are not fully supported, causing runtime errors. Metal acceleration in llama.cpp can conflict with other GPU usage on macOS.
Code Evidence
Automatic Faiss workaround from `ann/dense/faiss.py:10-15`:
if platform.system() == "Darwin" or os.name == "nt":
# Workaround for a Faiss issue causing segmentation faults. See txtai FAQ for more.
os.environ["OMP_NUM_THREADS"] = os.environ.get("OMP_NUM_THREADS", "1")
# Workaround for a Faiss issue with OMP: Error #15. See txtai FAQ for more.
os.environ["KMP_DUPLICATE_LIB_OK"] = os.environ.get("KMP_DUPLICATE_LIB_OK", "TRUE")
MPS device check with override from `models/models.py:160-169`:
@staticmethod
def hasmpsdevice():
return os.environ.get("PYTORCH_MPS_DISABLE") != "1" and torch.backends.mps.is_available()
llama.cpp Metal check from `pipeline/llm/llama.py:94-95`:
kwargs["n_gpu_layers"] = kwargs.get("n_gpu_layers", -1 if kwargs.get("gpu", os.environ.get("LLAMA_NO_METAL") != "1") else 0)
CI/CD macOS workaround from `.github/workflows/build.yml:37-40`:
echo "PYTORCH_MPS_DISABLE=1" >> $GITHUB_ENV
echo "LLAMA_NO_METAL=1" >> $GITHUB_ENV