Generative Adversarial Networks (GANs) Cheat Sheet

Explains the generator-discriminator minimax game, common failure modes like mode collapse, and a minimal PyTorch training loop for a GAN.

2 PagesAdvancedMar 5, 2026

Core Concepts

The adversarial training setup.

Generator (G)- Maps random noise z from a latent space to a synthetic sample meant to look real
Discriminator (D)- Binary classifier trained to distinguish real samples from G's fake samples
Minimax objective- min_G max_D E[log D(x)] + E[log(1 - D(G(z)))]; G and D are trained with opposing goals
Latent space- The input noise distribution (often standard normal) that G transforms into data
Mode collapse- G learns to produce only a few varieties of output that reliably fool D, losing diversity
Nash equilibrium- The theoretical training target where D can no longer distinguish real from fake (D outputs 0.5 everywhere)

Minimal GAN Training Loop

Alternating updates to the discriminator and generator.

python

import torch, torch.nn as nncriterion = nn.BCELoss()opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))for real_batch in dataloader:    batch_size = real_batch.size(0)    real_labels = torch.ones(batch_size, 1)    fake_labels = torch.zeros(batch_size, 1)    # --- Train Discriminator ---    z = torch.randn(batch_size, latent_dim)    fake_batch = G(z)    d_loss = criterion(D(real_batch), real_labels) + \             criterion(D(fake_batch.detach()), fake_labels)    opt_d.zero_grad(); d_loss.backward(); opt_d.step()    # --- Train Generator ---    g_loss = criterion(D(fake_batch), real_labels)  # wants D to say "real"    opt_g.zero_grad(); g_loss.backward(); opt_g.step()

Common GAN Variants

Architectures that address specific weaknesses of the vanilla GAN.

DCGAN- Uses convolutional/transposed-convolutional layers with batch norm for stable image generation
WGAN- Replaces the JS-divergence-based loss with the Wasserstein distance and weight clipping/gradient penalty for more stable training
Conditional GAN (cGAN)- Conditions both G and D on a label or class so generation can be controlled
CycleGAN- Learns unpaired image-to-image translation using cycle-consistency loss (no paired training data required)
StyleGAN- Injects latent style vectors at multiple resolutions for fine-grained control over generated image attributes

Stabilizing GAN Training

GANs are notoriously unstable to train.

Label smoothing- Use 0.9 instead of 1.0 for real labels to prevent an overconfident discriminator
Balance G and D capacity- If D becomes too strong too fast, G's gradients vanish and training stalls
Monitor both losses- A D loss near 0 usually signals D has overpowered G (or vice versa) -- losses should oscillate, not converge cleanly
Use Wasserstein loss for stability- WGAN-GP largely avoids mode collapse and vanishing gradients compared to the vanilla minimax loss

Pro Tip

Falling discriminator loss to near zero doesn't mean training is going well -- it usually means the discriminator has 'won,' generator gradients are vanishing, and you need to slow D down or switch to a Wasserstein-style loss.

Was this cheat sheet helpful?

Transfer Learning Cheat Sheet

Diffusion Models Cheat Sheet

Explore Topics

#GenerativeAdversarialNetworksGANs#GenerativeAdversarialNetworksGANsCheatSheet#DataScience#Advanced#CoreConcepts#Minimal#GAN#Training#MachineLearning#CheatSheet#SkillVeris

Sri Hayavadhana Info-Tech

Professional Web Designing Services