training
corpus & tokenization
corpus - training data
loss—
perplexity—
|grad|—
tokens0
sample generated while training
tokenization
vocabulary
learned merges
prompt, encoded
token frequency in corpus
transformer
—
value scale
−max 0 +max
weights
—tensors
selected tensor
architecture
hyper-parametersrebuild to apply
generate
input → outputinput prompt
output 0 tokens
next-token