Heuristic:Gretelai Gretel synthetics GPU Memory Allow Growth
| Knowledge Sources | |
|---|---|
| Domains | Optimization, Infrastructure |
| Last Updated | 2026-02-14 19:00 GMT |
Overview
TensorFlow GPU memory configuration that uses dynamic allocation (allow_growth) instead of pre-allocating all GPU memory, enabling multi-process and multi-model scenarios.
Description
By default, TensorFlow pre-allocates the entire GPU memory when it initializes. This prevents other processes or models from using the GPU simultaneously. The gretel-synthetics codebase explicitly sets `config.gpu_options.allow_growth = True` when loading models, which tells TensorFlow to allocate GPU memory incrementally as needed. This is critical for the parallel generation workflow where multiple worker processes may share the GPU, and for environments where other GPU workloads are running concurrently.
Usage
This heuristic is applied automatically when loading TensorFlow models via `_prepare_model()`. No user configuration is needed. This is particularly important when using the parallel generation feature, as each worker process needs to manage its own GPU memory allocation.
The Insight (Rule of Thumb)
- Action: Set `gpu_options.allow_growth = True` on TensorFlow session configs.
- Value: Memory allocated on-demand rather than pre-allocated.
- Trade-off: Slightly slower initial operations due to dynamic allocation, but allows GPU sharing between processes. May cause fragmentation on long-running sessions.
Reasoning
Pre-allocating all GPU memory is the TensorFlow default to reduce allocation overhead during training. However, for gretel-synthetics, model loading for generation happens separately from training, and the parallel generation system spawns multiple worker processes. Pre-allocation in any single process would starve others of GPU memory. The allow_growth setting trades a small amount of allocation overhead for much better multi-process compatibility.
Code Evidence
GPU memory configuration from `tensorflow/model.py:45-50`:
config = k.get_config()
# Don't pre-allocate memory, allocate as needed
config.gpu_options.allow_growth = True
k.set_session(tf.compat.v1.Session(config=config))