Few-Shot Learning

Definition

Few-shot learning is a machine learning approach in which a model learns to perform a new task from only a small number of labeled examples, rather than a large training dataset.

Overview

Traditional supervised learning typically requires thousands or millions of labeled examples to train a model on a task. Few-shot learning instead leverages a model that has already acquired broad knowledge — usually through large-scale pretraining — and adapts it to a new task using just a handful of examples, sometimes as few as one ("one-shot") or none at all ("zero-shot learning"). In the context of large language models, few-shot learning is most commonly achieved through prompting: a few example input-output pairs are included directly in the prompt, and the model uses attention mechanism-based pattern matching to infer the task without any weight updates at all. This is sometimes called "in-context learning" and is distinct from few-shot fine-tuning, where a small number of examples are actually used to update model weights. Few-shot learning is valuable because it drastically reduces the cost and time needed to adapt a model to a new task or domain, and it underlies much of how developers customize general-purpose LLMs for specific applications without full retraining. The Large Language Models course covers prompting and adaptation techniques like this in depth.

Key Concepts

• Adapts a pretrained model to a new task using only a few labeled examples
• Commonly implemented via in-context prompting for large language models
• Reduces labeling cost and time compared to full supervised training
• Related to but distinct from zero-shot and one-shot learning
• Relies on knowledge already captured during large-scale pretraining
• Can also refer to lightweight fine-tuning with a small example set

Use Cases

Frequently Asked Questions

Related in AI Fundamentals