Drag to rotateDouble-click to reset
Attention Is All You Need
From my notesCurrently reading
The lasting idea is architectural subtraction. Once attention can connect every position directly, recurrence is no longer a necessary organising principle—and an entire new model family becomes possible.
The paper that introduced the Transformer: an encoder-decoder architecture built entirely from self-attention and feed-forward layers, removing recurrence and making sequence training substantially more parallel.
Open original