Principle:Tensorflow Tfjs Weight Constraints
| Knowledge Sources | |
|---|---|
| Domains | Deep_Learning, Optimization, Regularization |
| Last Updated | 2026-02-10 06:00 GMT |
Overview
Hard constraints applied to weight tensors after each gradient update, projecting weights back to a valid region to enforce properties like bounded norms or non-negativity.
Description
Weight constraints differ from regularizers in that they enforce hard bounds rather than soft penalties. After each parameter update step, the constraints apply() method projects the weight tensor onto the feasible set. TensorFlow.js implements:
- MaxNorm: Clips weight norms to a maximum value per axis
- MinMaxNorm: Clips weight norms to a range [minValue, maxValue]
- NonNeg: Clips weights to be non-negative (sets negative values to 0)
- UnitNorm: Normalizes weights to unit norm along an axis
Constraints are specified per-layer via the kernelConstraint and biasConstraint configuration options.
Usage
Use constraints when weights must satisfy hard invariants, such as non-negative weights in certain physics-informed models, or bounded norms for training stability (e.g., maxNorm is common with recurrent layers to prevent gradient explosion).
Theoretical Basis
Pseudo-code Logic:
# After each gradient step:
weights = optimizer.update(weights, gradients)
weights = constraint.apply(weights) # Project back to feasible set
# MaxNorm example:
norms = norm(weights, axis)
desired = clip(norms, 0, max_value)
weights = weights * (desired / (norms + epsilon))