#moe

• 2026-04-04

  Gemma 4 26B: Google Drops a MoE Monster With 4B Active Params

  Google DeepMind released Gemma 4 today. The 26B A4B MoE scores 1441 on LMArena while burning just 4B active parameters. Here's what that means for local inference.

• 2026-04-02

  Llama 4 Scout: Meta's New MoE Monster, and It Already Runs Locally

  Meta just dropped Llama 4 Scout — 17B active parameters, 109B total, multimodal MoE with a 10M context window. I checked if it runs locally on AMD Strix Halo. Short answer: yes, today.

All tags →