#position-encoding - Tags - ML Learning Lab

4 posts · Transformer Series

Tag: #position-encoding

🗓 2026-04-09 • Transformer Series • ⏱ 49 min read

Length extrapolation in Transformers and LLMs: position encoding methods, RoPE extrapolation, PI, NTK-aware interpolation, YaRN, and Giraffe.

🗓 2026-04-05 • Transformer Series • ⏱ 47 min read

RoPE positional encoding, derivation, properties, extrapolation, and implementation.

🗓 2026-04-02 • Transformer Series • ⏱ 32 min read

APE vs RPE in Transformers: differences, representative methods, and relative-position design patterns.

🗓 2026-04-01 • Transformer Series • ⏱ 59 min read

Transformer positional encoding: why it is needed, design evolution, sinusoidal encoding analysis, and NoPE debates.