Hugging Face AI 开源博客：Your Experts Have Partners. There's HOPE for Keeping Them Together

Mixture-of-Experts (MoE) models have enabled state-of-the-art LLMs to be scaled to hundreds of billions of parameters, but they come with a fundamental and costly caveat: every expert MLP must occupy memory at inference…