In-Context Learning

Definition

In-context learning is the ability of a large language model to adapt its behavior on a new task using only examples or instructions provided in the prompt, without any gradient updates to the model's weights.

Overview

In-context learning (ICL) emerged as a striking property of large language models trained on next-token prediction over massive text corpora: given a handful of demonstrations of a task inside the prompt — such as a few input-output pairs for translation or classification — the model can infer the pattern and apply it to a new input, all within a single forward pass. This stands in contrast to the traditional machine learning workflow of fine-tuning, where a model's parameters are explicitly updated on labeled examples through backpropagation. ICL is typically described along a spectrum: zero-shot (only an instruction, no examples), one-shot (a single example), and few-shot (several examples). The phenomenon was popularized by the GPT-3 paper, which showed that scaling model and data size significantly improved few-shot performance without any weight updates. Researchers still debate the underlying mechanism — some frame it as implicit Bayesian inference over latent tasks seen during pretraining, others as a form of gradient descent simulated within the forward pass through attention layers. ICL is central to how most people interact with modern LLMs like GPT-4 or Claude: prompt engineering, few-shot prompting, and chain-of-thought prompting are all applications of in-context learning. It is also the foundation for retrieval-augmented generation, where retrieved documents are injected into the context so the model can reason over them without retraining. Because it requires no model updates, ICL makes LLMs dramatically cheaper and faster to adapt to new tasks, though it is bounded by the model's context window and can be less reliable than fine-tuning for highly specialized or high-stakes tasks.

Key Concepts

• Adapts model behavior purely through prompt content, with no weight updates
• Spans zero-shot, one-shot, and few-shot prompting styles
• Scales in effectiveness with model size and pretraining data diversity
• Bounded by the model's context window length
• Underpins prompt engineering techniques like chain-of-thought and few-shot prompting
• Enables rapid task adaptation without a training pipeline or labeled dataset
• Performance is sensitive to example order, phrasing, and selection
• Forms the mechanism that makes retrieval-augmented generation possible

Use Cases

Frequently Asked Questions

Related in AI Fundamentals