Recurrent Neural Networks Cheat Sheet

A reference for Recurrent Neural Networks covering LSTM and GRU implementations, sequence padding, vanishing gradients, and bidirectional architectures.

2 PagesIntermediateMar 5, 2026

LSTM in PyTorch

A sequence classifier built on an embedding and LSTM.

python

import torch.nn as nnclass LSTMClassifier(nn.Module):    def __init__(self, vocab_size, embed_dim, hidden_dim, num_classes):        super().__init__()        self.embedding = nn.Embedding(vocab_size, embed_dim)        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True, num_layers=2)        self.fc = nn.Linear(hidden_dim, num_classes)    def forward(self, x):        embedded = self.embedding(x)        output, (hidden, cell) = self.lstm(embedded)        return self.fc(hidden[-1])   # last layer's final hidden state

GRU in Keras

A lighter-weight recurrent layer for sequence classification.

python

from tensorflow.keras import layers, modelsmodel = models.Sequential([    layers.Embedding(input_dim=10000, output_dim=64),    layers.GRU(64, return_sequences=False),    layers.Dense(1, activation='sigmoid')])model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Sequence Padding

Normalize variable-length sequences to a fixed length.

python

from tensorflow.keras.preprocessing.sequence import pad_sequencespadded = pad_sequences(sequences, maxlen=100, padding='post', truncating='post')

Key Concepts

Core theory behind RNNs.

Hidden state- Carries information across time steps, updated at each step from the input and previous state
Vanishing/exploding gradients- Backpropagation through time over many steps can shrink or blow up gradients in vanilla RNNs
LSTM- Uses input, forget, and output gates plus a separate cell state to preserve long-range dependencies
GRU- Simplified gating (update and reset gates) with fewer parameters than LSTM, often comparable performance
Bidirectional RNN- Processes the sequence forward and backward, useful whenever full context is available upfront
Teacher forcing- Feeds the true previous token during training instead of the model's own prediction, stabilizing learning

Pro Tip

Reach for LSTMs or GRUs mainly when compute or context length is tightly constrained — Transformer-based architectures have largely superseded vanilla RNNs for tasks with longer-range dependencies because self-attention avoids the sequential bottleneck and vanishing-gradient issues RNNs face.

Was this cheat sheet helpful?

Convolutional Neural Networks Cheat Sheet

Transformers Architecture Cheat Sheet

Explore Topics

#RecurrentNeuralNetworks#RecurrentNeuralNetworksCheatSheet#DataScience#Intermediate#LSTMInPyTorch#GRUInKeras#SequencePadding#KeyConcepts#MachineLearning#CheatSheet#SkillVeris

Sri Hayavadhana Info-Tech

Professional Web Designing Services