Blogmark

Transformer is the core architecture behind modern Al, powering models like ChatGPT and Gemini. Introduced in 2017, it revolutionized how Al processes information. The same architecture is used for training on massive datasets and for inference to generate outputs. Here we use GPT-2 (small), simpler than newer ones but perfect for learning the fundamentals.

This explainer provides a nice set of text and visuals going through how the Transformer architecture works.

Here is the "Attention Is All You Need" paper that ushered in all these advances in large language models.