|

Exploring the Transformer Series (21) --- MoE

Mixture-of-Experts (MoE): conditional computation, routing, experts, load balancing, implementation, and parallel inference.