#length-extrapolation - Tags

3 posts · Transformer Series

Tag: #length-extrapolation

🗓 2026-04-09 • Transformer Series • ⏱ 49 min read

Length extrapolation in Transformers and LLMs: position encoding methods, RoPE extrapolation, PI, NTK-aware interpolation, YaRN, and Giraffe.

🗓 2026-04-05 • Transformer Series • ⏱ 47 min read

RoPE positional encoding, derivation, properties, extrapolation, and implementation.

🗓 2026-04-01 • Transformer Series • ⏱ 59 min read

Transformer positional encoding: why it is needed, design evolution, sinusoidal encoding analysis, and NoPE debates.