Jamba
Jamba is a family of large language models from AI21 Labs that combines the Transformer architecture with a state-space model (Mamba) architecture in a hybrid design, aimed at improving efficiency and context handling compared to pure Transformer models.
6 resources across 1 library
Glossary Terms(6)
Jamba (AI21)
Jamba is a family of large language models from AI21 Labs that combines the Transformer architecture with a state-space model (Mamba) architecture in a hybrid…
State Space Model (Mamba)
A State Space Model (SSM), in the context of modern sequence modeling, is a neural architecture — with Mamba as a prominent example — that processes sequences…
Jurassic (AI21)
Jurassic is a family of large language models developed by AI21 Labs, an Israeli AI company, with Jurassic-1 released in 2021 and Jurassic-2 following in 2023,…
Jamba
Jamba is a family of large language models from AI21 Labs built on a hybrid architecture combining Transformer and Mamba (state space model) layers with mixtur…
Mixture of Experts
Mixture of Experts (MoE) is a neural network architecture that replaces a single dense feed-forward layer with many smaller "expert" sub-networks and a learned…
DBRX
DBRX is an open-weight large language model released by Databricks (developed by its Mosaic AI research team) in March 2024, built as a fine-grained mixture-of…