What is a foundation model?

It's a large model trained on broad data that can be adapted to many downstream tasks, rather than being built from scratch for a single narrow purpose.

Where did the term come from?

It was introduced by Stanford's Center for Research on Foundation Models in a 2021 paper describing the trend toward single large pre-trained models being adapted for many uses.

Are all LLMs foundation models?

Most large language models are foundation models, but the foundation model concept also covers non-text systems, including image, audio, and multimodal models.

How are foundation models adapted for specific tasks?

Common approaches include fine-tuning on task-specific data, prompt engineering, retrieval augmentation, and adding lightweight adapters — all of which avoid retraining the entire model from scratch.

What's the difference between open and closed foundation models?

Open-weight models (such as Llama) can be downloaded and run or fine-tuned by anyone, while closed models (such as GPT-5 or Claude) are accessed only through a provider's API.

Why are foundation models risky to build responsibly?

Their scale, broad capability, and wide downstream use mean flaws such as bias, factual errors, or safety gaps can propagate into many applications, which is why alignment and evaluation work matters.

How expensive is it to train a foundation model?

Training frontier foundation models can cost tens to hundreds of millions of dollars in compute, which is why most organizations adapt existing foundation models rather than training new ones from scratch.

AI Models

Foundation Model

BeginnerConcept11K learners

A foundation model is a large-scale model pre-trained on broad, diverse data — typically using self-supervised learning — that can be adapted to a wide range of downstream tasks through fine-tuning, prompting, or additional training,…

#FoundationModel #AIModels#Concept#Beginner#LLM #GenerativeAI #SmallLanguageModel #FineTuning #ArtificialIntelligence#Glossary#SkillVeris

Definition

Overview

The term was popularized by a 2021 Stanford paper describing a shift in AI development: instead of training a separate model for every task, organizations increasingly train one very large, general-purpose model on massive datasets and then adapt it for many purposes. Models like GPT-5, Claude, Gemini, and Llama are all foundation models — trained once on huge text (and often image, audio, or code) corpora, then specialized through fine-tuning, retrieval augmentation, or simply through carefully engineered prompts. What makes a model a 'foundation' is its generality and adaptability rather than any single architecture. Most foundation models today use the Transformer architecture and are trained with self-supervised objectives — such as predicting the next token — on internet-scale text, so no manual labeling of every example is required. Because they encode broad statistical patterns of language, reasoning, and (increasingly) other modalities, the same base model can be redirected toward chat assistants, coding copilots, embedding generation, classification, and more. Foundation models underpin nearly the entire modern AI stack: cloud providers offer them as APIs (via services like Amazon Titan or Azure OpenAI), enterprises fine-tune them for internal use cases, and open-weight releases let developers self-host and adapt them freely. Their scale and generality bring real risks too — they can be expensive to train, opaque in their reasoning, and prone to encoding biases present in their training data — which is why responsible deployment practices and AI Safety work have become central to the field. Courses like Large Language Models cover how these models are trained, adapted, and deployed in practice.

Key Concepts

Pre-trained once on broad, large-scale data rather than per-task from scratch
Adaptable to many downstream tasks via fine-tuning, prompting, or retrieval
Typically built on Transformer-based architectures at large parameter scale
Trained with self-supervised objectives that don't require manually labeled data
Can span multiple modalities — text, images, audio, and code
Available both as open-weight releases and closed, API-only offerings
Serves as the base layer for chatbots, copilots, agents, and embedding systems

Use Cases

Base model behind consumer assistants such as ChatGPT, Claude, and Gemini

Starting point for enterprise fine-tuning on proprietary or domain-specific data

Backbone for retrieval-augmented generation and enterprise knowledge assistants

Source model for distillation into smaller, deployable Small Language Models

General-purpose reasoning and coding engine inside developer tools

Multimodal foundation for image, video, and audio generation products

Frequently Asked Questions

From the Blog

Data Science

NumPy for Data Science: Arrays and Vectorisation

NumPy is the foundation of Python's scientific computing stack. This guide covers ndarrays, vectorised operations, broadcasting, linear algebra, and why NumPy is 10-100x faster than equivalent Python loops — with practical examples for data science work.