Transformer Architecture
The Transformer is a neural network architecture built entirely around the self-attention mechanism, allowing it to model relationships between all elements of an input sequence in parallel rather than processing them one at a time.
6 resources across 1 library
Glossary Terms(6)
Perplexity (NLP)
Perplexity is a metric for evaluating language models that measures how well a model predicts a sample of text, calculated as the exponentiated average negativ…
Transformer Architecture
The Transformer is a neural network architecture built entirely around the self-attention mechanism, allowing it to model relationships between all elements of…
Encoder-Decoder Model
An encoder-decoder model is a neural network architecture split into two components — an encoder that processes an input sequence into an internal representati…
Sequence-to-Sequence Model
A sequence-to-sequence (seq2seq) model is a class of neural network models designed to transform an input sequence into an output sequence, where the two seque…
Word2Vec
Word2Vec is a family of neural network-based techniques, introduced by Google researchers in 2013, that learn dense vector representations (word embeddings) of…
GloVe
GloVe (Global Vectors for Word Representation) is a word embedding technique, developed at Stanford, that learns dense vector representations of words by facto…