Llama

Meta ships two structurally different model lines

The selection mistake on this page is treating "Meta" as one shortlist entry. Meta currently ships two structurally different lines that almost never compete for the same workload, and the quality gap between them is large enough that confusing the two is a real decision risk.

• Muse Spark is the closed-API frontier reasoning line. Meta branded, not Llama branded, closed weights, served via API. Quality Score 95.6 at rank #20 of 197 evaluated variants, Arena ELO rank #7 of 174, rank #6 of 93 on HLE. That puts it in the same competitive bracket as GPT-5 and Claude Opus 4 and makes it the only Meta model currently in the closed-API frontier tier.
• Llama is the open-weights line. Llama 4 (Maverick, Scout) is the current generation, ~1 year old at the time of this writing and carrying Meta's first MoE architecture. Llama 3 (8B, 70B, 405B) is the previous generation, still in active production use at organisations that built on it. Pick a Llama variant when open-weights flexibility (data residency, fine-tune, air-gapped inference, run-anywhere) is on the requirement list at all. The entire Llama line sits below the closed-API quality frontier in our index, so "Llama" alone is not a frontier-quality argument; on shared per-benchmark evals Llama 4 is the strongest Meta open-weights tier today.

If you are not sure which line applies, the question is whether open-weights matters. If yes, you want Llama and the picking criterion is hardware fit, not quality leadership. If no, you want Muse Spark.

Where Llama sits today (with a caveat on Quality Score)

The full open-weights line sits well below the closed-API frontier in our index. Read these numbers as relative positioning across our leaderboard, not as a verdict on which Meta release is "best":

• Llama 4 Maverick lands at Quality Score 59.1, rank #174 of 197.
• Llama 4 Scout lands at 59.6, rank #169 of 197.
• Llama 3 70B (3.3) lands at 56.9, rank #180 of 197.
• Llama 3 405B (3.1) lands at 56.6, rank #181 of 197.
• Llama 3 8B (3.1) lands at 44.6, rank #194 of 197.

Caveat worth reading before quoting these ranks: Quality Score is a percentile-based composite across whichever benchmarks a model has results on. Coverage is uneven across the Llama line. Llama 4 was evaluated on harder modern benchmarks (HLE, ARC-AGI-2, GPQA Diamond) where absolute scores are low across the whole field, which drags the composite percentile down. Llama 3 was not re-run on most of those, so its composite is computed from an easier benchmark mix. On every benchmark where both lines have a result, Llama 4 Maverick outperforms Llama 3 70B 3.3 (GPQA Diamond 69.8 vs 50.5, MMLU Pro

vs 68.9, Arena ELO

vs 1318.0, LiveBench

vs no Llama 3 70B result). Read the per-benchmark variant table on this page before pinning a production decision to a composite score: the underlying benchmarks tell the more useful story.

Meta announced Llama 4 Behemoth as a frontier-class checkpoint but has not publicly released it. Until weights or a hosted route ship, treat Behemoth as a marketing reference, not a usable option. The variant is intentionally absent from the table below.

How to pick a Llama variant (open-weights only)

If open-weights is the requirement, the picking criterion is hardware fit and architecture, with per-benchmark depth on top.

• Hosted-API or large self-host, current-generation MoE: Llama 4 Maverick ($0.2 input / $0.8 output per 1M, 1M-token context). Beats Llama 3 70B 3.3 on every benchmark where both have results in our index (GPQA Diamond, MMLU Pro, Arena ELO, LiveBench). The Quality Score composite reads lower than Llama 3 70B 3.3 because Maverick was evaluated on more hard benchmarks, not because Llama 3 70B 3.3 is the stronger model.
• Smaller MoE footprint: Llama 4 Scout ($0.1 / $0.3, 327K context). Reach for Scout when Maverick's quality lift does not justify the larger memory footprint or per-token cost.
• Dense 70B self-host with the broadest ops experience: Llama 3 70B 3.3. Hosted ~$0.13 / $0.4, dense 70B parameters, the most-recent Llama 3 checkpoint Meta shipped. Defensible when your deployment is built around a dense 70B and a Llama 4 migration would require MoE serving changes you don't want yet, or when your evals were qualified on this specific checkpoint.
• Single-GPU self-host, broadest tooling, lowest capability ceiling: Llama 3 8B. Hosted ~$0.05 / $0.08, ~16K context, most battle-tested fine-tune ecosystem in the entire open-weights world. Quality ceiling is genuinely low (rank #194 of 197). Pick only when predictable single-GPU deployment plus the tooling depth outweigh the capability gap.
• Very-large dense deployment: Llama 3 405B 3.1. The largest dense Llama Meta has shipped. Sized for organisations that built their stack on it and have not migrated. There is no Llama 4 dense variant at this size; Llama 4 went MoE instead.

The combination of "frontier reasoning quality" and "open weights" is not in the Llama line at all. If that combination is the requirement, look at DeepSeek-R1 or the Qwen3 MoE variants instead. Both sit above the Llama line on the harder modern reasoning benchmarks at comparable or lower hosted-API pricing.

Muse Spark: variant choice and data gaps

Muse Spark currently lists a single Thinking variant in our index. Benchmark coverage is dense (Quality Score 95.6 ranking #20 of 197, Arena ELO 1488.0 ranking #7 of 174, HLE 42.8 ranking #6 of 93, SWE-Bench Verified 77.4, ARC-AGI-2 42.5, GPQA Diamond 89.5). That is top-tier closed-API competition.

Where the data is weak: hosted-API pricing and the context-window limit Meta exposes for Muse Spark are not yet in our index. Treat the listed score data as definitive and the deployment economics as something to confirm directly against Meta's API pricing page before committing to the line for production. We will backfill once the provider publishes a stable hosted route.

Where this family is the wrong call

• You need open weights AND frontier reasoning in the same variant. Muse Spark gets you the quality but is closed; nothing in the Llama line clears the quality bar for frontier reasoning. DeepSeek-R1 or Qwen3 MoE are the open-weights frontier today. Llama is not.
• Procurement requires a US-jurisdiction-only model with frontier performance. Muse Spark clears this gate; Llama clears the jurisdiction gate but not the frontier-performance one. Both are usable choices where Qwen3 or DeepSeek would not be (that is structural to Meta's positioning, not a benchmark statement).
• Behemoth. Meta announced Llama 4 Behemoth but has not publicly released it. The variant is intentionally absent from the table below; we will add it when it ships as a usable hosted or open-weights route, not when it appears in marketing.

Where the data is currently weak

• Muse Spark hosted pricing and exposed context window are missing from our index, as noted above. Cross-check Meta's API docs before you commit to the line.
• Llama 3 70B and 405B benchmark coverage is thinner than the Llama 4 line because most newer evaluations did not re-run on the previous generation. Treat the listed scores on those rows as directional. The Quality Score composite is sensitive to which benchmarks each variant has been tested on; that sensitivity is the reason Llama 3 70B 3.3 reads above Llama 4 on the composite even though Llama 4 wins every shared per-benchmark eval. Read the per-benchmark variant table before pinning a production decision to a single composite score.
• Series-level Pareto positioning (one chart spanning Llama 4 + Muse Spark on quality vs. cost) is not yet in our pipeline; the per-variant table on this page is the load-bearing artifact.
• Pricing changes faster than our scrape cadence. If you are making a procurement decision, cross-check pricing against the provider's own docs before you commit.

Sources worth reading

• Llama 4 official model page: release notes, intended use, licence terms
• Llama on Hugging Face: model cards, weights, licence file
• Provider docs: AWS Bedrock pricing: one of the canonical hosted-API pricing references for Llama variants