Robots Atlas>ROBOTS ATLAS
Llama 3.1 405B
AI Models›Llama

Llama 3.1 405B

3.1 405B · Family: Llama
Open-weight dense language model by Meta AI with 405 billion parameters, 128K token context window, and tool use support, released July 23, 2024.
✓ Active✓ Public access⚖ Open weightsLLMTool-using model📁 Llama
Context window
128K tokenów
tokens
Parameters
405B
parameters
Release date
23 July 2024
🏢Meta AIProducer
Access:APIDownloadDeployment:☁ Cloud💻 Local

Overview

Llama 3.1 405B is the largest and most capable dense language model in the Llama 3.1 family, developed by Meta AI and released on July 23, 2024, as an open-weights model under the Llama 3.1 Community License.

Specification

The model has 405 billion parameters. Context window: 128,000 tokens. Knowledge cutoff: December 2023. Supports tool use and fine-tuning. Available in both a pre-trained (base) and instruction-tuned variant (Llama 3.1 405B Instruct).

Benchmark Results (Instruct)

According to the official Meta model card on Hugging Face: MMLU 88.6% (5-shot), MMLU-Pro 73.3%, GPQA 50.7% (0-shot), MATH 73.8% (0-shot CoT), HumanEval 89.0% (0-shot Pass@1), GSM8K 96.8%, DROP 84.8% (3-shot F1, pre-trained base).

Availability

Weights are publicly available on Hugging Face (meta-llama/Llama-3.1-405B and -Instruct). The model can be run locally (requires a datacenter-class GPU cluster, e.g., NVIDIA H100/A100) or accessed through API providers, including Meta, Azure AI, AWS, Google Cloud, Databricks, Groq, and Together AI.

Classification
LLMTool-using model
Family: Llama
Access & deployment
APIDownload
CloudLocal
Weights: Open weights
Key parameters
📏 Context: 128K tokenów
🧩 Parameters: 405B
✓ Tools · ✓ Fine-tuning
📥 Input: text, image

Technical specification

Context window
128K tokenów
tokens
Parameters
405B
parameters
License
Llama 3.1 Community License
Hardware requirements
No official minimum requirements are specified. Local deployment requires a datacenter-class GPU cluster (e.g., NVIDIA H100 / A100).
Features:✓ Tool use✓ Fine-tuning
Modalities
⬇ Input
textimage
⬆ Output
textcodestructured_data

Capabilities and applications

Native model capabilities
Reasoning
The model's ability to reason logically and solve complex problems.
Category: reasoning
Multi-step reasoning
Carrying out multi-step chains of reasoning across long, complex tasks.
Category: reasoning
Long context
Maintaining coherence and focus across very long input context.
Category: language
Coding
Generating, analysing and modifying source code.
Category: coding
Function Calling
Category: planning
Structured output
Producing data in structured formats such as JSON.
Category: structured_generation
Image understanding
Analysing and interpreting the content of images.
Category: vision
Chart understanding
Reading and interpreting charts, tables and diagrams.
Category: vision
OCR
Recognising text within images and documents.
Category: vision
Multilingual
Understanding and generating text in many languages.
Category: language
Planning
Forming and executing action plans for complex tasks.
Category: planning
Streaming output
Category: reasoning

Benchmark results

7 benchmarks
MMLU
88.6
📄 Meta AI – Llama 3.1 Technical Report
GSM8K
96.8
📄 Meta AI – Llama 3.1 Technical Report
HumanEval
89.0
📄 Meta AI – Llama 3.1 Technical Report
MATH
73.8
📄 Meta AI – Llama 3.1 Technical Report
GPQA
0-shot accuracy · Instruction-tuned model (Llama 3.1 405B Instruct), 0-shot
50.7%
📄 Meta Llama 3.1 model card (Hugging Face), July 2024
MMLU-Pro
5-shot accuracy · Instruction-tuned model, 5-shot
73.3%
📄 Meta Llama 3.1 model card (Hugging Face), July 2024
DROP
3-shot F1 · Pre-trained base model (Llama 3.1 405B Base)
84.8%
📄 IBM analysis referencing Meta Llama 3.1 model card, July 2024
Result applies to the base (pre-trained) model, not the instruction-tuned variant.

Sources and related pages