Transformer Systems · Skill Path

Attention From Scratch

A focused transformer path for implementing attention scores, self-attention, masks, and multi-head shape transforms.

Transformer Systems 4 steps intermediate to advanced 7.2 hr 1 review decks
  1. 01
    Attention Mechanism Lesson · Foundations And Data Flow · 1.5 hr · Reading
  2. 02
    Self-Attention Mechanics Lesson · Attention And Positional Information · 2.5 hr · Reading
  3. 03
    Padding, Causal, And Packed Masks Lesson · Attention And Positional Information · 1.7 hr · Review deck
    Flashcard deck
  4. 04
    Multi-Head Self-Attention Lesson · Attention And Positional Information · 1.5 hr · Reading