Dolly (model)
Dolly is an open-source instruction-following large language model released by Databricks, notable as one of the first LLMs whose instruction-tuning dataset was fully human-generated and released under a fully permissive commercial license.
Definition
Dolly is an open-source instruction-following large language model released by Databricks, notable as one of the first LLMs whose instruction-tuning dataset was fully human-generated and released under a fully permissive commercial license.
Overview
Dolly was released by Databricks in two versions during early 2023. Dolly 1.0 was a proof-of-concept showing that a smaller, older base model (EleutherAI's GPT-J 6B) could be fine-tuned on a modest set of instruction examples generated using the Self-Instruct technique with outputs from a larger model, and still exhibit surprisingly ChatGPT-like instruction-following behavior. It was primarily a demonstration of how accessible this kind of fine-tuning had become rather than a production-ready model, and its training data included outputs derived from a non-commercially-licensed source, which limited its own licensing. Dolly 2.0, released shortly after, addressed that limitation directly: Databricks employees hand-wrote roughly 15,000 instruction-response pairs — the "databricks-dolly-15k" dataset — covering categories like brainstorming, classification, closed and open QA, summarization, and creative writing, with no reliance on outputs from a proprietary model like GPT-3/4. This made both the dataset and the resulting fine-tuned model (based on EleutherAI's Pythia 12B) available under a fully permissive license (Apache 2.0/CC BY-SA), free for unrestricted commercial use — a notable contrast to many contemporaneous instruction-tuned models whose training data derived from closed-model outputs and carried non-commercial restrictions as a result. Dolly was never positioned as a state-of-the-art model in absolute quality; Databricks was explicit that it was released to demonstrate that useful instruction-following behavior could be elicited from a modestly sized open base model with a comparatively small, high-quality, and license-clean dataset, rather than requiring either massive scale or restrictively licensed synthetic data. Its significance was largely catalytic: the databricks-dolly-15k dataset became a widely reused building block in the open-source LLM ecosystem for training and evaluating other instruction-tuned models, independent of whether Dolly itself remained in active use.
Key Concepts
- One of the first LLMs with a fully human-generated, license-clean instruction dataset
- databricks-dolly-15k: ~15,000 hand-written instruction-response pairs across multiple task categories
- Dolly 2.0 fine-tuned from EleutherAI's Pythia 12B base model
- Released under a fully permissive license (Apache 2.0 / CC BY-SA) allowing unrestricted commercial use
- Demonstrated instruction-following capability without relying on outputs from closed proprietary models
- Developed and released by Databricks
- Dataset became a widely reused resource independent of the Dolly model itself
- Positioned as a research/demonstration release rather than a state-of-the-art model