Glossario IA

📖

termini

Masked Language Model (MLM)

Training objective where the model predicts randomly masked words in a sentence, forcing the model to understand the bidirectional context of language.

📖

termini

Sequence to Sequence Encoding (Seq2Seq)

Neural network architecture composed of two parts, an encoder that processes an input sequence and a decoder that generates an output sequence, used for translation and text synthesis.

📖

termini

Layer Normalization

Regularization technique that stabilizes the training of deep neural networks by normalizing the activations of each layer, accelerating convergence and improving performance.

📖

termini

Subword Tokenization

Tokenization method that divides words into smaller units (subwords), allowing to manage a finite vocabulary while representing rare or unknown words by concatenation of known subwords.