Transformer

In June 2017, a team of eight researchers from Google Brain and Google Research — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser and Illia Polosukhin — published the foundational paper "Attention Is All You Need" at the NeurIPS conference (Neural Information Processing Systems). This paper introduced the Transformer architecture, which completely abandoned the recurrent approaches based on LSTM and GRU networks that had dominated since the early 2010s. The Transformer rests on a central mechanism: multi-head attention, which enables the model to process an entire sequence simultaneously by computing relevance relationships between each of its elements, in parallel rather than sequentially. The architecture comprises an encoder and decoder, each consisting of stacked layers containing self-attention units and feed-forward neural networks. The attention mechanism uses three learnable matrices — Query (Q), Key (K) and Value (V) — to compute a relevance score between each element and all others. Unlike recurrent networks that process data token by token, the Transformer processes the entire sequence in parallel, which dramatically accelerates training and efficiently exploits modern GPU architectures. The Transformer's success quickly exceeded initial expectations. In 2018, Google published BERT (Bidirectional Encoder Representations from Transformers), which achieved state-of-the-art performance on eleven natural language processing tasks. The same year, OpenAI launched GPT (Generative Pre-trained Transformer), demonstrating that Transformers trained on massive unannotated text corpora could generate coherent and sophisticated text. The architecture became the absolute foundation of all large language models: GPT-2, GPT-3, GPT-4, BERT, T5, Gemini, Claude. The decade 2017–2027 sees the Transformer reshape the global technology industry. Practical applications span conversational assistants, machine translation, code generation, text analysis and artistic creation. The Transformer marks the opening of the era of generative artificial intelligence, whose economic, social and cultural implications continue to unfold.