Principle:Tensorflow Tfjs Noise Regularization

Knowledge Sources	Tensorflow_Tfjs Dropout as Bayesian Approximation
Domains	Deep_Learning, Regularization
Last Updated	2026-02-10 06:00 GMT

Overview

Regularization technique that adds random noise to layer activations during training to prevent overfitting and improve model robustness.

Description

Noise layers inject stochasticity into the forward pass during training, forcing the network to learn more robust representations that are invariant to small perturbations. TensorFlow.js implements:

• GaussianNoise: Adds zero-mean Gaussian noise with configurable standard deviation
• GaussianDropout: Multiplies activations by random values drawn from a Gaussian with mean 1
• AlphaDropout: Specialized dropout for SELU-activated networks that maintains self-normalizing properties

All noise layers are active only during training and pass data through unchanged during inference.

Usage

Use noise layers as an alternative or complement to standard Dropout for regularization. GaussianNoise is effective when inputs are continuous-valued. AlphaDropout should be used specifically with SELU activations to maintain the self-normalizing property.

Theoretical Basis

$y=x+ϵ,ϵ\sim {𝒩}(0,{\sigma }^2)\text{(GaussianNoise)}$

$y=x\cdot ϵ,ϵ\sim {𝒩}(1,\sqrt{\frac{p}{1-p}})\text{(GaussianDropout)}$

Related Pages

• Implementation:Tensorflow_Tfjs_Noise_Layers