Gemma-2-9B-it

Gemma-2-9B-it is an open, instruction-tuned large language model (LLM) with 9 billion parameters, developed by Google DeepMind and released in June 2024. It is a decoder-only, text-to-text model designed for tasks such as question answering, summarization, reasoning, and code generation.

The Gemma 2 architecture combines alternating local (sliding-window) and global attention layers, Grouped-Query Attention (GQA), and logit soft-capping in the attention and final layers. The 9B variant was trained using knowledge distillation from a larger teacher model. The model uses an 8192-token context window.

The model was trained on 8 trillion tokens using JAX and ML Pathways on TPU hardware (TPUv5p). The -it variant underwent supervised instruction tuning (SFT) and reinforcement learning from human feedback (RLHF). Weights are released under the Gemma license (requires acceptance of Google's terms); the model can be downloaded and run locally or in the cloud.