Expert-Parallelism on 111qqz的小窝https://111qqz.com/tags/expert-parallelism/Recent content in Expert-Parallelism on 111qqz的小窝Hugo -- gohugo.iozh-cnhust.111qqz@gmail.com (111qqz)hust.111qqz@gmail.com (111qqz)© 2011-2026 111qqzTue, 03 Nov 2026 22:30:00 +0800Expert Parallelism：为什么让 Token 去找 Expert，而不是搬运参数？https://111qqz.com/2026/11/expert-parallelism-distributed-moe/Tue, 03 Nov 2026 22:30:00 +0800hust.111qqz@gmail.com (111qqz)https://111qqz.com/2026/11/expert-parallelism-distributed-moe/<p><a href="https://111qqz.com/2026/11/mixture-of-experts-router-token-dispatch-grouped-gemm/" >上一篇</a>把单个 MoE Layer 的执行流走了一遍：Router 选 Expert、Token Dispatch 按 Expert 重排、Grouped GEMM 统一调度、Combine 加权合回原始 Token 顺序。整条链路假设所有 Expert 都在同一张 GPU 上。</p>