03·Gemini quirks: grounding, multimodal, caching, task-aware embeddings

Introduction

A lesson on Google Gemini and Vertex AI specifics: system_instruction as a separate field instead of a role, Google Search grounding with compliance, multimodal limits (video 1fps, audio 32kHz), safety_settings with four categories, explicit context caching, AI Studio vs Vertex AI, function calling with OpenAPI 3.0, JSON mode with response_schema, the code_execution tool, the candidates response structure with a finishReason field (including RECITATION), gemini-2.0-flash-thinking with visible reasoning, cumulative streaming, the SentencePiece tokeniser, 2M context and NIAH degradation, generation_config as a nested object with a top_k field, embeddings with task_type, model versioning (-002 aliases), error codes, Batch API with a 50% discount, multimodal Live API WebSocket, supervised fine-tuning on Vertex, Model Garden with third-party models, Vertex grounding with Search/Maps/your own data, and enterprise compliance (IAM, VPC, CMEK, audit logs, CMEK, BAA).