Building a GPTLesson 3 of 11
Keeping the signal: residuals and layer norm
Add each update to the stream instead of replacing it, then rescale to mean 0 and variance 1. [2, 4, 6, 8] becomes ±1.34 and ±0.45.
Locked
Unlock the rest of Deep Learning: from one neuron to a transformer.
- Every lesson, every resource, unlocked instantly.
- Track progress and pick up where you left off.
- Free preview lessons stay readable from the outline.