CNN architectures — VGG, ResNet, EfficientNet — Deep Learning & Neural Networks

The history of CNN architectures is a history of solving the vanishing gradient problem at scale. VGG (2014) showed that depth — using only 3×3 convolutions stacked 16–19 layers deep — dramatically outperforms shallow networks with large filters. But VGG's purely sequential stack hit a ceiling: beyond 19 layers, gradients vanished before reaching early layers and accuracy degraded. ResNet (2015) solved this with a single elegant idea: skip connections that add the layer's input directly to its output, creating a highway for gradients to bypass problematic layers. With skip connections, ResNet successfully trained 152-layer networks and won ImageNet by a large margin. EfficientNet (2019) took a different approach: instead of depth alone, it scales width (channels), depth (layers), and resolution (input size) simultaneously using a compound coefficient derived from neural architecture search. EfficientNet-B7 achieves state-of-the-art accuracy with 8.4x fewer parameters than ResNet-152, making it the practical choice for most production deployments.

Analogy🏏Cricket

🏏 Think of it like cricket: The Duckworth-Lewis-Stern (DLS) method, the ICC bowling economy metric, and the net run rate each measure team performance but optimise for completely different things — DLS cares about wickets-in-hand, economy cares about runs-per-over, NRR cares about tournament-wide margins. Using the wrong metric to evaluate a bowler is like using MSE for classification: technically computable but optimising the wrong thing entirely. Just as you would never rank a spinner by batting average, you should never use MSE when your model predicts probabilities. Just as focal DLS adjustments down-weight easy chases and amplify close finishes, focal loss down-weights easy examples and amplifies hard minority-class examples. The choice of scoring metric defines what excellence means — and so does the choice of loss function.