Vellum
By Vellum
Vellum is a platform for building, testing, and deploying applications powered by large language models, offering tools for prompt engineering, model comparison, evaluation, and production monitoring of LLM workflows.
Definition
Vellum is a platform for building, testing, and deploying applications powered by large language models, offering tools for prompt engineering, model comparison, evaluation, and production monitoring of LLM workflows.
Overview
Vellum targets engineering and product teams building LLM-powered features, providing a workspace to draft and iterate on prompts, compare outputs across different models and providers, and run structured evaluations before shipping changes to production. This helps teams avoid regressions when tweaking prompts or switching between models like those from OpenAI or other providers. Beyond prompt experimentation, Vellum offers workflow-building tools for chaining multiple LLM calls and tools together (similar in spirit to frameworks like LangChain), along with production monitoring so teams can track latency, cost, and output quality once a feature is live. Vellum sits in the broader "LLMOps" category alongside tools like LangSmith, aimed at making LLM application development more reliable and observable, closer to established software engineering practices.
Key Features
- Prompt engineering workspace with version history
- Side-by-side comparison of outputs across different LLMs
- Structured evaluation tooling to test prompts before deployment
- Visual workflow builder for chaining LLM calls and tools
- Production monitoring for latency, cost, and output quality
- Collaboration features for teams iterating on AI features together