Kong extends its gateway to agent-to-agent traffic with the AI A2A Proxy plugin -- if you already run Kong for APIs, LLMs, and MCP, adding A2A routing keeps all traffic governance in one place.

Agentgateway is the most complete open-source agent gateway -- it natively speaks A2A and MCP, handles all three gateway layers (LLM, MCP, agent), and is now a Linux Foundation project.

APISIX is the plugin-driven alternative -- if you already run APISIX for API management, its AI plugins add LLM proxying, smart routing, and token rate limiting without adopting a new gateway.

Envoy AI Gateway is the CNCF-backed open-source option for AI traffic management -- built on the battle-tested Envoy proxy, it brings LLM routing, credential management, and inference-aware load balancing to Kubernetes-native deployments.

Helicone is what you pick when observability is your primary concern and you want a fast, lightweight gateway that doubles as your LLM monitoring layer -- Rust-based, open-source, and deployable as a single binary.

Cloudflare AI Gateway is the easiest on-ramp if you're already on Cloudflare -- add a URL prefix and you get caching, analytics, and rate limiting for free. The tradeoff: SaaS-only, no self-hosting, and limited advanced features.

LiteLLM is the default choice when you need a self-hosted, open-source LLM proxy with zero licensing cost -- it does one thing well (unified LLM API) and stays out of your way.

Portkey is the most feature-complete purpose-built LLM gateway -- it combines routing, guardrails, observability, and prompt management in one platform, which is both its strength and its vendor lock-in risk.

If you already run Kong for API management, Kong AI Gateway is the path of least resistance to enterprise LLM governance -- it extends what you have rather than adding another proxy layer.

Standard DevSecOps assumes deterministic systems. AI systems break all three assumptions -- outputs are non-deterministic, vulnerabilities can be in the prompt or model, and rollback may mean reverting a prompt, a model version, or a guardrail configuration.