Flash Attention
FlashAttention is an IO-aware exact attention algorithm that reorders and fuses the computation of transformer self-attention to minimize reads and writes to GPU high-bandwidth memory, delivering significant speedups and memory savings without changing the mathematical result.
6 resources across 1 library
Glossary Terms(6)
Vector Embedding Model
A vector embedding model is a neural network trained to map discrete inputs — words, sentences, images, or other objects — into dense numerical vectors such th…
Sparse Attention
Sparse attention is a family of transformer attention mechanisms that restrict each token to attending only to a subset of other tokens, rather than the full s…
Flash Attention
FlashAttention is an IO-aware exact attention algorithm that reorders and fuses the computation of transformer self-attention to minimize reads and writes to G…
Rotary Positional Embedding
Rotary Positional Embedding (RoPE) is a technique for injecting token position information into transformer attention by rotating query and key vectors in mult…
KV Cache
The KV cache (key-value cache) is a memory structure used during autoregressive transformer inference that stores the key and value projections of previously p…
Grouped Query Attention
Grouped Query Attention (GQA) is a transformer attention variant in which multiple query heads share a single set of key and value heads, reducing the size of…