Principle:Tensorflow Tfjs Embedding Representation
| Knowledge Sources | |
|---|---|
| Domains | Deep_Learning, NLP, Representation_Learning |
| Last Updated | 2026-02-10 06:00 GMT |
Overview
Learnable lookup table that maps discrete integer indices (tokens, categories) to dense continuous-valued vectors, enabling neural networks to process categorical inputs.
Description
The Embedding layer maintains a weight matrix of shape [inputDim, outputDim] where each row corresponds to a dense vector representation for one input category. Given an integer index, the layer performs a simple table lookup to retrieve the corresponding row. The vectors are learned during training via backpropagation.
Embeddings are fundamental to NLP (word/token embeddings), recommendation systems (user/item embeddings), and any task with high-cardinality categorical inputs. They replace inefficient one-hot encodings with compact, meaningful vector representations where similar items cluster together in the embedding space.
Usage
Use the Embedding layer as the first layer for any input that consists of integer indices (token IDs, category IDs). Configure inputDim to the vocabulary/category size and outputDim to the desired embedding dimensionality. Optionally set maskZero: true to mask padding tokens in sequence models.
Theoretical Basis
The embedding is equivalent to a one-hot encoding followed by a linear layer, but implemented as a direct lookup for efficiency:
Pseudo-code Logic:
# Equivalent operations:
output = one_hot(input, vocab_size) @ W # Matrix multiply (slow)
output = W[input] # Direct lookup (fast, used in practice)