Training a deep CNN from random initialisation requires millions of labelled images and days of GPU compute. Transfer learning is the technique of using a model pre-trained on a large dataset (typically ImageNet's 14 million images across 1,000 classes) as the starting point for a new task, dramatically reducing both data and compute requirements. The underlying principle is feature reuse: the early layers of any CNN trained on natural images learn universal features — edges, textures, colour gradients — that are useful for virtually any image recognition task, from medical imaging to sports analytics. Fine-tuning extends transfer learning by unfreezing some or all of the pre-trained layers and continuing to train them on the new dataset at a very low learning rate, allowing task-specific adaptation while preserving the broad visual knowledge. The combination of a pre-trained feature extractor with a task-specific head is now the standard approach for any computer vision task with fewer than 1 million labelled training images — which is the vast majority of real-world applications.
35 minadvanced
Transfer learning and fine-tuning
Analogy🏏Cricket
🏏 Think of it like cricket: The Duckworth-Lewis-Stern (DLS) method, the ICC bowling economy metric, and the net run rate each measure team performance but optimise for completely different things — DLS cares about wickets-in-hand, economy cares about runs-per-over, NRR cares about tournament-wide margins. Using the wrong metric to evaluate a bowler is like using MSE for classification: technically computable but optimising the wrong thing entirely. Just as you would never rank a spinner by batting average, you should never use MSE when your model predicts probabilities. Just as focal DLS adjustments down-weight easy chases and amplify close finishes, focal loss down-weights easy examples and amplifies hard minority-class examples. The choice of scoring metric defines what excellence means — and so does the choice of loss function.
Lesson 15 of 35
0% complete