GPT-3
By OpenAI
GPT-3 is a 175-billion-parameter decoder-only Transformer language model released by OpenAI in 2020, which demonstrated that scaling a language model far beyond prior efforts produced strong few-shot and zero-shot task performance purely…
Definition
GPT-3 is a 175-billion-parameter decoder-only Transformer language model released by OpenAI in 2020, which demonstrated that scaling a language model far beyond prior efforts produced strong few-shot and zero-shot task performance purely through prompting, without any task-specific fine-tuning.
Overview
GPT-3 extended the approach behind GPT-2 by roughly two orders of magnitude in parameter count, and the increase in scale produced a qualitative shift in capability. Where earlier models needed fine-tuning on labeled examples to perform well on a given task, GPT-3 could often perform a new task competently just by being shown a handful of examples in its prompt ("few-shot learning") or sometimes with no examples at all ("zero-shot"), simply by describing the task in natural language. GPT-3 was trained on a broad mixture of filtered Common Crawl data, WebText2, Books corpora, and Wikipedia, using the same decoder-only, next-token-prediction objective as its predecessors. OpenAI released it not as downloadable weights but as a paid API, a distribution model that became the template for how many subsequent frontier models would be commercialized — access through hosted inference rather than open release. GPT-3's release in 2020 catalyzed the modern wave of applied generative AI: it powered early products for copywriting, coding assistance, and chatbots, and its in-context learning behavior — adapting to new tasks from prompt examples alone — became a foundational concept underpinning prompt engineering as a discipline. GPT-3 was later refined into GPT-3.5, which used instruction tuning and reinforcement learning from human feedback to become the model underlying the original public release of ChatGPT in November 2022. GPT-3's scale also sharpened debate around the costs and risks of very large models — energy use, bias amplification from web-scale training data, and the concentration of capability in organizations with sufficient compute — discussions that continue to shape how frontier labs describe and govern their models.
Key Features
- 175 billion parameters, roughly 100x larger than GPT-2
- Strong few-shot and zero-shot performance via in-context learning
- Trained on filtered Common Crawl, WebText2, books, and Wikipedia data
- Released as a hosted API rather than downloadable open weights
- Directly refined into GPT-3.5, the original model behind ChatGPT
- Popularized prompting and in-context learning as core interaction paradigms