Memory and attentionLesson 1 of 9Free preview
Memory that fades
Pass the clue along word by word: the RNN pull on it vanishes to 6% after five. The LSTM forget gate keeps 73%. Attention looks straight back.
Memory and attentionLesson 1 of 9Free preview
Pass the clue along word by word: the RNN pull on it vanishes to 6% after five. The LSTM forget gate keeps 73%. Attention looks straight back.