Memory and attentionLesson 2 of 9Free preview
Attention: every word asks the others
Each word scores every other word and spreads 100% of its attention by softmax. "na" puts 80% on "pathaben", the verb it flips.
Memory and attentionLesson 2 of 9Free preview
Each word scores every other word and spreads 100% of its attention by softmax. "na" puts 80% on "pathaben", the verb it flips.