Meta’s latest family of artificial intelligence models, Llama 4, brings significant innovations to multimodal model development. In addition to the two models currently available—Llama 4 Scout and Llama 4 Maverick—a very powerful model called Llama 4 Behemoth is in development. Behemoth is expected to play a significant role in STEM-related tasks (science, technology, engineering, and mathematics) in the future.
Recently, many multimodal models have been introduced. These models can process and integrate different types of data—such as text, images, sound, and video—at the same time. This allows them to understand questions in a richer context and solve much more complex problems than previous text-only models. However, this strength can also be a drawback, as such models usually need far more resources than traditional single-modal systems. Meta addresses this issue with the Mixture of Experts (MoE) architecture used in the Llama 4 family. The MoE approach activates only a portion of the model for a given input, which improves efficiency and greatly reduces computational costs. This method is not unique to Llama 4; many large companies are following a similar trend. Nevertheless, Llama 4’s open-source strategy clearly sets it apart from its competitors.
As mentioned earlier, only the two smaller models in the family—Scout and Maverick—are currently available. Both models have 17 billion active parameters, meaning that they process input using 17 billion parameters. However, each model actually contains many more total parameters. Scout has 109 billion parameters, while Maverick has 400 billion. This is due to the MoE architecture: the models activate specific submodules (called “experts” by Meta) when processing input. Accordingly, Scout uses 16 experts and Maverick uses 128 experts. Although Scout is smaller than Maverick, it has the unique feature of a context window that can handle up to 10 million tokens, making it ideal for analyzing long texts, documents, or large codebases. While Maverick does not support such an extensive context window, several benchmarks show that it outperforms competitors like GPT-4o or Gemini 2.0 Flash in inference and coding tasks, even while using only half as many parameters as DeepSeek V3.
Although still in development, Meta claims that Behemoth will perform better than GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro in STEM-related tasks. Like its two smaller siblings, Behemoth will have 288 billion active parameters, but with 16 submodules it will have nearly 2,000 billion parameters in total. Behemoth is also notable because Meta plans to use this model to train smaller models, and it could be integrated into Meta services such as Messenger, Instagram Direct, and WhatsApp.
