Weight Decay
Weight decay is a regularization technique that adds a penalty proportional to the magnitude of a model's weights to the loss function, discouraging overly large parameter values and reducing overfitting.
3 resources across 1 library
Glossary Terms(3)
Gradient Clipping
Gradient clipping is a training technique that caps the magnitude of gradients during backpropagation, preventing the exploding-gradient problem that can desta…
Weight Decay
Weight decay is a regularization technique that adds a penalty proportional to the magnitude of a model's weights to the loss function, discouraging overly lar…
Early Stopping
Early stopping is a regularization technique that halts model training once performance on a held-out validation set stops improving, preventing the model from…