Attention Is All You Need Annotated Paper

Attention Is All You Need

One of the most game changing paper of recent times. This paper introduces the transformer architecture and talks about the self attention mechanism.

Please feel free to read along the paper with my notes and highlights.

Color	Meaning
Green	Topics about the current paper
Yellow	Topics about other relevant references
Blue	Implementation details/ maths
Red	Text including my thoughts, questions, and understandings

CITATION

@misc{vaswani2017attention,
      title={Attention Is All You Need}, 
      author={Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin},
      year={2017},
      eprint={1706.03762},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}