Convolutional Neural Networks Cheat Sheet
A cheat sheet for Convolutional Neural Networks covering PyTorch and Keras implementations, convolution and pooling operations, and transfer learning.
2 PagesIntermediateMar 8, 2026
CNN in PyTorch
A minimal convolution-pool-convolution-pool classifier.
python
import torchimport torch.nn as nnclass SimpleCNN(nn.Module): def __init__(self, num_classes=10): super().__init__() self.conv1 = nn.Conv2d(3, 32, kernel_size=3, padding=1) self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1) self.pool = nn.MaxPool2d(2, 2) self.fc = nn.Linear(64 * 8 * 8, num_classes) def forward(self, x): x = self.pool(torch.relu(self.conv1(x))) # 32x32 -> 16x16 x = self.pool(torch.relu(self.conv2(x))) # 16x16 -> 8x8 x = x.flatten(1) return self.fc(x)
CNN in Keras
The same architecture using the Keras Sequential API.
python
from tensorflow.keras import layers, modelsmodel = models.Sequential([ layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)), layers.MaxPooling2D((2, 2)), layers.Conv2D(64, (3, 3), activation='relu'), layers.MaxPooling2D((2, 2)), layers.Flatten(), layers.Dense(10, activation='softmax')])
Transfer Learning
Fine-tune a pretrained backbone on a new task.
python
import torch.nn as nnimport torchvision.models as modelsbackbone = models.resnet50(weights='IMAGENET1K_V2')for param in backbone.parameters(): param.requires_grad = False # freeze pretrained weightsbackbone.fc = nn.Linear(backbone.fc.in_features, num_classes) # replace the classifier head
Key Concepts
Core theory behind CNNs.
- Convolution- Slides learnable filters (kernels) across the input to detect local patterns like edges and textures
- Feature map- Output of applying one filter across the input; stacked feature maps form a layer's output
- Pooling- Downsamples feature maps (max or average) to shrink spatial size and add translation invariance
- Stride & padding- Stride sets the filter's step size; padding ('same'/'valid') controls the output spatial dimensions
- Receptive field- Region of the input that influences a given output unit; grows with network depth
- Transfer learning- Reuse a pretrained backbone (ResNet, EfficientNet, etc.) and fine-tune it on a new task with less data
Computing Output Spatial Dimensions
The formula behind how kernel size, stride, padding, and dilation determine a conv layer's output shape.
python
def conv_output_size(input_size, kernel_size, stride=1, padding=0, dilation=1): effective_kernel = dilation * (kernel_size - 1) + 1 return (input_size + 2 * padding - effective_kernel) // stride + 1# Example: 224x224 input, 7x7 kernel, stride 2, padding 3 (ResNet stem)out = conv_output_size(224, kernel_size=7, stride=2, padding=3) # -> 112# 'same' padding in Keras auto-computes padding so output size == input size / stride# PyTorch's Conv2d needs it computed manually unless padding='same' is passed (stride=1 only)
Depthwise Separable Convolutions
Factorize a standard convolution into per-channel spatial filtering plus a 1x1 channel-mixing step, drastically cutting parameters (MobileNet-style).
python
import torch.nn as nnclass DepthwiseSeparableConv(nn.Module): def __init__(self, in_ch, out_ch, kernel_size=3, stride=1): super().__init__() self.depthwise = nn.Conv2d( in_ch, in_ch, kernel_size, stride=stride, padding=kernel_size // 2, groups=in_ch # groups=in_ch -> one filter per channel ) self.pointwise = nn.Conv2d(in_ch, out_ch, kernel_size=1) # mixes channels def forward(self, x): return self.pointwise(self.depthwise(x))# Standard conv: in_ch * out_ch * k * k params# Depthwise separable: in_ch * k * k + in_ch * out_ch params -- ~8-9x fewer for k=3
Grad-CAM for Model Interpretability
Visualize which spatial regions of an image most influenced a CNN's prediction.
python
import torchactivations, gradients = {}, {}def fwd_hook(module, inp, out): activations['value'] = outdef bwd_hook(module, grad_in, grad_out): gradients['value'] = grad_out[0]target_layer = model.layer4[-1] # last conv block, e.g. in a ResNettarget_layer.register_forward_hook(fwd_hook)target_layer.register_full_backward_hook(bwd_hook)output = model(image.unsqueeze(0))class_score = output[0, predicted_class]model.zero_grad()class_score.backward()# Global-average-pool the gradients to get per-channel importance weightsweights = gradients['value'].mean(dim=(2, 3), keepdim=True)cam = torch.relu((weights * activations['value']).sum(dim=1)).squeeze()
Augmentation Pipeline with torchvision.transforms.v2
Modern augmentation stacks that generalize better than basic flips/crops alone.
python
from torchvision.transforms import v2train_transform = v2.Compose([ v2.RandomResizedCrop(224, scale=(0.7, 1.0)), v2.RandomHorizontalFlip(p=0.5), v2.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2), v2.RandAugment(), # policy-based combination of augmentations v2.ToDtype(torch.float32, scale=True), v2.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), v2.RandomErasing(p=0.25), # cutout-style occlusion regularization])
Architectural Patterns Beyond the Basics
Ideas that recur across modern CNN backbones.
- Residual connections- Skip connections that add a block's input to its output, letting gradients bypass layers and enabling much deeper networks (ResNet)
- 1x1 convolutions- Used purely to change channel depth (up or down) without touching spatial dimensions; cheap way to mix or bottleneck features
- Dilated (atrous) convolution- Inserts gaps between kernel elements to enlarge the receptive field without adding parameters or downsampling, common in segmentation models
- Global average pooling- Replaces flatten+dense classifier heads with a per-channel spatial average, cutting parameters and reducing overfitting
- Feature pyramid network- Combines feature maps from multiple depths/scales so detectors can recognize both small and large objects
- Squeeze-and-excitation blocks- Learn per-channel attention weights from global context, letting the network re-weight feature maps adaptively
- Discriminative fine-tuning- Use progressively smaller learning rates for earlier (more generic) layers than later (more task-specific) layers when fine-tuning
Pro Tip
When fine-tuning a pretrained CNN on a small dataset, freeze the early convolutional layers, which learn generic edges and textures, and only unfreeze the later layers plus the classification head — this cuts overfitting risk and speeds up training.
Was this cheat sheet helpful?
Explore Topics
#ConvolutionalNeuralNetworks#ConvolutionalNeuralNetworksCheatSheet#DataScience#Intermediate#CNNInPyTorch#CNNInKeras#TransferLearning#KeyConcepts#MachineLearning#CheatSheet#SkillVeris
