Exploring the Transformer Series (12) --- Multi-head Self-Attention
Multi-head self-attention in Transformers: motivation, principles, implementation details, and modern head-composition improvements.
Tag: #qkv
Exploring the Transformer Series (10) --- Self-Attention
Self-attention in Transformers: principles, implementation details, scaling/softmax analysis, and modern optimization directions.