Before deep learning existed, machine learning relied on hand-crafted features — engineers would manually decide which aspects of data mattered. The perceptron, invented by Frank Rosenblatt in 1958, was the first computational model that could learn from data by adjusting its own internal weights. However, a single perceptron can only solve linearly separable problems — it cannot learn XOR, let alone recognise handwritten digits. The solution was to stack perceptrons into layers, creating a network where each layer learns increasingly abstract representations. Without activation functions, stacking layers is mathematically pointless — multiple linear transformations collapse into a single linear transformation, making depth useless. Activation functions introduce the non-linearity that makes deep networks capable of approximating any function.
35 minadvanced
Perceptron, activation functions and layers
Analogy🏏Cricket
🏏 Think of it like cricket: The Duckworth-Lewis-Stern (DLS) method, the ICC bowling economy metric, and the net run rate each measure team performance but optimise for completely different things — DLS cares about wickets-in-hand, economy cares about runs-per-over, NRR cares about tournament-wide margins. Using the wrong metric to evaluate a bowler is like using MSE for classification: technically computable but optimising the wrong thing entirely. Just as you would never rank a spinner by batting average, you should never use MSE when your model predicts probabilities. Just as focal DLS adjustments down-weight easy chases and amplify close finishes, focal loss down-weights easy examples and amplifies hard minority-class examples. The choice of scoring metric defines what excellence means — and so does the choice of loss function.
Lesson 1 of 35
0% complete