Principle:Tensorflow Tfjs Weight Regularization

Knowledge Sources	Tensorflow_Tfjs Regularization Techniques
Domains	Deep_Learning, Optimization, Regularization
Last Updated	2026-02-10 06:00 GMT

Overview

Technique that adds penalty terms to the loss function based on weight magnitudes, discouraging overly complex models and reducing overfitting.

Description

Weight regularization constrains the magnitude of model weights during training by adding a penalty term to the loss function. This encourages the model to learn simpler patterns that generalize better to unseen data. TensorFlow.js implements three standard regularizers:

• L1 (Lasso): Adds the sum of absolute weight values, encouraging sparsity
• L2 (Ridge): Adds the sum of squared weight values, encouraging small weights
• L1L2 (Elastic Net): Combines both L1 and L2 penalties

Regularizers are applied per-layer via the kernelRegularizer, biasRegularizer, and activityRegularizer configuration options.

Usage

Apply weight regularization when training models that show signs of overfitting (high training accuracy but low validation accuracy). L2 is the most common default choice. L1 is preferred when feature selection or sparse models are desired.

Theoretical Basis

$\text{L1 penalty}={\lambda }_1\sum _i|w_i|$

$\text{L2 penalty}={\lambda }_2\sum _i{w}_i^2$

$\text{Total Loss}=\text{Data Loss}+\text{L1 penalty}+\text{L2 penalty}$

Related Pages

• Implementation:Tensorflow_Tfjs_Regularizers
• Implementation:Tensorflow_Tfjs_Exports_Regularizers