What Is a Neural Network?
A neural network is a machine learning model composed of layers of interconnected nodes, called neurons, loosely inspired by the way biological neurons transmit signals. Each neuron computes a weighted sum of its inputs, adds a bias term, and passes the result through a nonlinear activation function. By stacking many such neurons into layers, and stacking many layers on top of each other, neural networks can approximate extremely complex, nonlinear functions, which is why they underlie modern advances in image recognition, natural language processing, and many other domains.
Cricket analogy: A single fielder reacts to a ball by weighing distance, angle, and pace before deciding to dive; stack eleven such fielders across multiple lines of the field and the team can respond to almost any shot, however complex.
Layers: Input, Hidden, and Output
A typical feedforward neural network has an input layer, whose size matches the number of input features; one or more hidden layers, where the actual representation learning happens; and an output layer, whose size and activation depend on the task — a single sigmoid neuron for binary classification, a softmax layer for multi-class classification, or a single linear neuron for regression. A network with more than one hidden layer is often called a 'deep' neural network, and the field of deep learning is largely about designing and training such multi-layer architectures effectively.
Cricket analogy: A team's structure has openers matching the number of overs available (input), middle-order batters who build the actual innings (hidden layers doing the real work), and a finisher whose role depends on the format — steady accumulation for Tests, big hitting for T20; more finishing specialists in the order is like a deeper batting lineup.
Weights, Biases, and the Forward Pass
Each connection between neurons has an associated weight, and each neuron has a bias. During the forward pass, an input vector flows through the network layer by layer: at each neuron, inputs are multiplied by their corresponding weights, summed together with the bias, and passed through an activation function such as ReLU or sigmoid. The final layer's output is the network's prediction. Weights and biases start out randomly initialized and are the parameters that training adjusts — via gradient descent and backpropagation — so that the network's predictions increasingly match the true labels in the training data.
Cricket analogy: Each partnership between two batters has a run rate, and each batter has a natural risk tolerance; as the innings flows over-by-over, shots are chosen by weighing the required rate against risk and passed through a decision like 'attack' or 'defend', with the final score being the outcome.
import numpy as np
def relu(z):
return np.maximum(0, z)
def sigmoid(z):
return 1 / (1 + np.exp(-z))
# A tiny two-layer network: 3 inputs -> 4 hidden units -> 1 output
np.random.seed(0)
W1 = np.random.randn(3, 4) * 0.1
b1 = np.zeros(4)
W2 = np.random.randn(4, 1) * 0.1
b2 = np.zeros(1)
def forward(x):
z1 = x @ W1 + b1
a1 = relu(z1) # hidden layer activation
z2 = a1 @ W2 + b2
a2 = sigmoid(z2) # output layer activation (binary classification)
return a2
x_sample = np.array([[0.5, -1.2, 0.3]])
prediction = forward(x_sample)
print("Predicted probability:", prediction)The Universal Approximation Theorem states that a feedforward network with even a single hidden layer of sufficient width can approximate any continuous function on a bounded domain to arbitrary precision — this is why neural networks are so flexible, though in practice deeper (not just wider) networks tend to learn efficient representations more effectively.
Without a nonlinear activation function, stacking multiple layers is mathematically equivalent to a single linear layer, no matter how many layers you add — the composition of linear functions is still linear. Nonlinearity (ReLU, sigmoid, tanh, etc.) is what actually gives neural networks their expressive power.
Why Depth and Nonlinearity Matter
Each layer of a neural network can be thought of as learning progressively more abstract features from the previous layer's output — in an image classifier, early layers might detect edges, middle layers might detect shapes or textures, and later layers might detect entire objects. This hierarchical feature learning, combined with nonlinear activations that let the network model complex decision boundaries, is what distinguishes neural networks from simpler linear models like logistic regression, which can only separate classes with a straight line or hyperplane.
Cricket analogy: A young player first learns to grip the bat correctly (edges), then to time a basic forward defense (shapes), and only years later reads bowlers' hands to anticipate late swing and play match-winning innings (objects) — well beyond what a beginner following a simple straight-bat rule could ever achieve.
- A neural network stacks layers of neurons, each computing a weighted sum plus bias, passed through a nonlinear activation function.
- Networks have an input layer, one or more hidden layers, and an output layer whose shape depends on the task.
- The forward pass computes predictions by propagating inputs through the layers using the current weights and biases.
- Nonlinear activation functions are essential; without them, stacked layers collapse into an equivalent single linear layer.
- Deeper layers tend to learn progressively more abstract, higher-level features.
- Training adjusts weights and biases via gradient descent and backpropagation to minimize prediction error.
Practice what you learned
1. What does a single neuron in a neural network compute before applying its activation function?
2. Why is a nonlinear activation function necessary between layers of a neural network?
3. What does the Universal Approximation Theorem state?
4. What determines the appropriate activation function and size of a neural network's output layer?
5. In the analogy of hierarchical feature learning in an image classifier, what do early layers typically learn compared to later layers?
Was this page helpful?
You May Also Like
Activation Functions
Activation functions inject non-linearity into neural networks, letting them approximate complex functions instead of collapsing into a single linear transformation.
Training with Backpropagation
Backpropagation efficiently computes how much each weight in a neural network contributed to the error, using the chain rule to propagate gradients backward from output to input.
Gradient Descent Explained
Understand how gradient descent iteratively adjusts model parameters to minimize a loss function, and how learning rate and variants like SGD affect convergence.
Logistic Regression Explained
Introduces logistic regression as a classification algorithm that models class probability via the sigmoid function, despite its regression-sounding name.