Batch Normalization
Batch normalization is a neural network training technique that normalizes the inputs to each layer using the mean and variance computed over a mini-batch, then rescales and shifts the result with learnable parameters.
6 resources across 2 libraries
Glossary Terms(5)
Batch Normalization
Batch normalization is a neural network training technique that normalizes the inputs to each layer using the mean and variance computed over a mini-batch, the…
Layer Normalization
Layer normalization is a neural network normalization technique that normalizes activations across the features of a single training example, rather than acros…
Dropout Regularization
Dropout is a regularization technique that randomly deactivates a fraction of a neural network's neurons during each training step, preventing the network from…
Learning Rate Scheduling
Learning rate scheduling is the practice of systematically adjusting a model's learning rate during training, rather than keeping it fixed, to speed up converg…
Gradient Accumulation
Gradient accumulation is a training technique that simulates a larger batch size than fits in available memory by summing gradients over several smaller mini-b…