Agentgateway

Agentgateway is an open-source, high-performance gateway under the Apache 2.0 license, hosted by the Linux Foundation as an AAIF (Agentic AI Infrastructure Foundation) project. The project was donated by Solo.io (known for Istio Ambient Mesh and Gloo Mesh). Innovation: one Rust binary, one control plane, one data plane handling all traffic types at once - service traffic (HTTP, gRPC, TCP), LLM provider traffic, MCP tool calls, agent-to-agent (A2A). Replaces a chain of separate gateways (API gateway + LLM gateway + MCP gateway + service mesh) with one coherent layer. Contributors: Microsoft, Alibaba, Adobe, AWS, Cisco, Salesforce, Huawei, Amdocs, T-Mobile, UBS. Installation: curl -sL https://agentgateway.dev/install | bash (binary), Docker ghcr.io/agentgateway/agentgateway:latest, or a Helm chart on Kubernetes (Gateway API).

Six main features: (1) Service Gateway - HTTP/2, gRPC, TCP, north-south and east-west traffic; observable by default; zero-config TLS 1.3, OIDC, mTLS rotation; native Envoy compatibility. (2) LLM Gateway - drop-in OpenAI-compatible API for 12+ backends (OpenAI, Anthropic, Bedrock, Gemini, Vertex, Cohere, Mistral, DeepSeek, Ollama, xAI, Perplexity, Replicate, Azure OpenAI, Hugging Face, NVIDIA NIM, Databricks, Snowflake, Meta/Llama); per-team token budgets, semantic caching, prompt redaction, credentialing, automatic failover. (3) Inference Routing - smart routing for self-hosted GPU pools; latency-aware, cost-aware, model-aware; plug-in vLLM, TGI, Triton - Agentgateway picks the warmest replica.

Key features continued: (4) MCP Gateway - control plane for the Model Context Protocol. Discovery, signing, scoping, observability for every agent tool call. Treats MCP servers as microservices - with versioning, RBAC, and audit trails. Critical for enterprise AI deployments - central visibility over all tools used by agents. (5) A2A Gateway - native Agent-to-Agent protocol support. Bridge between LangChain, CrewAI, Google ADK, Claude Code, Cursor, Codex, Hermes Agent, OpenClaw, Pi.dev agents. Identity, tracing, replay. (6) Security & Observability - OpenTelemetry by default; per-tool, per-agent, per-tenant counters; OPA-evaluated allow/deny on every hop; tamper-evident audit logs; PII shield. Native integrations with Prometheus, Datadog, Sentry, Langfuse, W&B, PagerDuty. Kubernetes CRDs via Helm chart. Foundation for an 'AI Mesh' in the organization.

Ecosystem: supported agent frameworks - LangChain, LangGraph, CrewAI, Google ADK, Hermes Agent, OpenClaw, kagent, NemoClaw. Supported tools/IDEs - Claude Code, Cursor, Codex, VS Code, OpenCode, Pi.dev, GitHub CLI. MCP server integrations - GitHub MCP, Slack MCP, Postgres MCP, Docker MCP. Infrastructure - Kubernetes, AWS, Google Cloud, GKE, HashiCorp Vault. Observability - OpenTelemetry, Prometheus, Datadog, Sentry, Langfuse, Weights & Biases, PagerDuty. CNCF-scale contributor base: engineers from Microsoft, AWS, Google, Adobe, Cisco, Salesforce, Alibaba. Dell CTO John Roese, CoreWeave SVP Chen Goldberg, Akamai SVP Jon Alexander, Microsoft CNCF Ambassador Mitch Connors, T-Mobile Director Rob Hansen, UBS Executive Director Sathish Krishnan, and Kyverno CEO Jim Bugwadia have publicly confirmed the project's strategic importance.

Market position: Agentgateway enters the AI-native gateway/mesh segment, competing with: LiteLLM Proxy (BerriAI, self-hosted Python), Portkey, Helicone AI Gateway, the Envoy + custom filters layer. Differentiators: (a) governance-first with Linux Foundation/AAIF - enterprise trust; (b) one Rust binary instead of a chain of components; (c) native support for both traditional traffic (HTTP/gRPC/TCP) and AI-native (LLM/MCP/A2A); (d) native Kubernetes CRDs; (e) OpenTelemetry and OPA built-in. kagent support (from the same Solo.io team) - agent orchestration on Kubernetes. Roadmap Q3-Q4 2026: semantic caching improvements, cross-agent memory federation, deeper policy engine.