Heuristic:Tensorflow Tfjs Memory Management With Tidy
| Metadata | |
|---|---|
| Source | Experience |
| Domains | Optimization, Memory_Management |
| Date | 2026-02-10 |
Overview
Use tf.tidy() to wrap operations for automatic cleanup of intermediate tensors, preventing memory leaks in TensorFlow.js applications.
Description
TensorFlow.js does not have automatic garbage collection for GPU-backed tensors. Without explicit cleanup, every tensor operation creates a new tensor that persists in GPU/CPU memory. tf.tidy() automatically disposes all intermediate tensors created within its scope, except the return value. For tensors that need to persist across tidy blocks, use tf.keep().
Usage
Use this heuristic in any TensorFlow.js application, whether browser or Node.js. Apply whenever performing repeated tensor operations (inference loops, training steps, data preprocessing). Especially critical in long-running applications.
The Insight
- Action: Wrap all tensor operations in tf.tidy(() => { ... }) blocks
- Value: Automatically disposes ALL intermediate tensors except the return value
- Trade-off: Cannot use async functions inside tidy (use manual dispose for async flows)
- Additional: Use tf.keep() to preserve specific tensors from tidy cleanup. Variables do NOT auto-dispose inside tidy—use tf.disposeVariables() separately.
- Monitor: Use tf.memory() to check numTensors, numBytes, numBytesInGPU during development
Reasoning
GPU memory is finite and not garbage-collected. Each tensor operation allocates new GPU memory. In a typical inference loop running 60fps, this would leak hundreds of tensors per second. tf.tidy() is implemented via a scope mechanism in the Engine class (tfjs-core/src/engine.ts) that tracks all tensor allocations and disposes non-returned tensors when the scope exits.
Code Evidence
From tfjs-core/src/globals.ts:150-193
const y = tf.tidy(() => {
const one = tf.scalar(1);
const a = tf.scalar(2);
const b = a.square();
return b.add(one);
});
// one, a, b are automatically disposed. y survives.
From docs/OPTIMIZATION_PURE_GPU_PIPELINE.md:36-38
// Remember to dispose the texture after use, otherwise, there will be memory leak.
data.tensorRef.dispose();