Comparison of the routing mechanisms in MoE models including DeepSeek, Qwen, Mixtral, Llama 4, and gpt-oss
A comparison reviewed the routing configurations for 10 open-source MoE models: the number of experts ranged from 8 in Mixtral to 384 in Kimi K2 and DeepSeek-V4-Pro, while top-k varied from 1 in Llama 4 Maverick to 8.
Return to original text
A short introduction is retained here; the full text from the source website is not reposted. For specific wording and subsequent updates, please refer to the original article.
Open AIHOT original