Proxy (X-Forwarded-For) mode
This guide walks through setting up Envoy Gateway in a cluster where you
front the OpenStack load balancer with your own upstream proxy (for
example a CDN, WAF, or edge proxy) that terminates the client connection
and injects the real client IP into the X-Forwarded-For header. The
OpenStack load balancer itself stays in TCP passthrough; the upstream
proxy is what carries the client IP for you.
This mode requires your own upstream proxy. It is only correct when a CDN, WAF, or edge proxy that you operate sits in front of the load balancer and injects
X-Forwarded-For. Without one, no real client IP ever reaches Envoy, so your backends see only the load balancer’s internal IP. If clients connect straight to Elastx with nothing in front, use direct (PROXY-protocol) mode instead; it carries the client IP for you and needs no proxy.
Note: Your
ClientTrafficPolicymust setclientIPDetection.xForwardedForwithnumTrustedHopsset to the number of trusted proxies in front of Envoy. Without it Envoy will not honour the incomingX-Forwarded-Forheader and your access logs and rate limiting will see the load balancer’s internal IP.
Note: The upstream proxy must terminate TLS to inject
X-Forwarded-For; it can only read and modify headers on decrypted traffic. It then opens a fresh connection to the load balancer (which passes it through untouched) and Envoy terminates TLS again on theGatewayHTTPS listener. A pure TCP/TLS passthrough upstream cannot inject the header.
If you are not sure which variant applies to your cluster, see the Envoy Gateway overview.
The shared-Gateway layout
A cluster runs one shared Gateway in a dedicated namespace that serves
routes from all your application namespaces through a single load balancer
and IP. Each application namespace opts in with a label and contributes its own
HTTPRoutes. This is the standard setup, the same single-entry-point model
ingress-nginx gave you, where one controller fronted every host.
%%{init: {'theme':'base','themeVariables':{'primaryColor':'#DAE7EC','primaryBorderColor':'#1E343E','primaryTextColor':'#1E343E','lineColor':'#5A7A8A','clusterBkg':'#EEF3F6','clusterBorder':'#9BB3BF','edgeLabelBackground':'#FFFFFF'}}}%%
flowchart TB
client(["Clients"]):::client --> proxy["Your upstream proxy<br/>CDN / WAF / edge<br/>injects X-Forwarded-For"]:::proxy
proxy -->|TCP passthrough| lb["OpenStack load balancer<br/>TCP mode · one LB · one IP"]:::lb
lb --> gw
subgraph gwns["gateway namespace"]
gw["Gateway 'shared'"]:::gw
ctp["ClientTrafficPolicy<br/>xForwardedFor.numTrustedHops: 1"]:::policy
cert["TLS certificate<br/>team-a.example.com"]:::policy
end
ctp -.->|attaches to| gw
cert -.->|terminates TLS| gw
subgraph ta["team-a namespace (labelled)"]
appa["app + HTTPRoute"]:::app
end
subgraph tb["team-b namespace (labelled)"]
appb["app + HTTPRoute"]:::app
end
gw -->|"team-a.example.com"| appa
gw -->|"team-b.example.com"| appb
classDef client fill:#FFFFFF,stroke:#1E343E,color:#1E343E;
classDef proxy fill:#CFE8FF,stroke:#0041C2,color:#1E343E;
classDef lb fill:#DAE7EC,stroke:#1E343E,color:#1E343E;
classDef gw fill:#FBBD18,stroke:#1E343E,stroke-width:2px,color:#1E343E;
classDef policy fill:#F5F8FA,stroke:#1E343E,color:#1E343E;
classDef app fill:#DAE7EC,stroke:#1E343E,color:#1E343E;
What the shared Gateway gives you:
- One load balancer per cluster: a single LB and IP front all your teams, the way a single ingress controller did before, and your upstream proxy points at a single origin IP.
- One
ClientTrafficPolicyto manage, in thegatewaynamespace, with TLS terminated there (one certificate per hostname you serve). - Self-service for app teams: an app team only labels its namespace and
creates an
HTTPRoute; it never touches the shared Gateway.
Each
Gatewayprovisions its own load balancer. Running more than one (a separateGatewayand load balancer for a single namespace, viaallowedRoutes.namespaces.from: Same) is a non-standard setup, for the rare case that genuinely needs an isolated IP or blast radius.
Prerequisites
- A dedicated namespace for the shared Gateway. The examples use
gateway. - One or more application namespaces. The examples use
team-a. - DNS for each public hostname (the examples use
team-a.example.com, replace with your own) pointing at your upstream proxy (CDN / WAF / edge proxy), not directly at the load balancer. If it resolves straight to the load balancer, traffic bypasses the proxy and noX-Forwarded-Foris injected. - Your upstream proxy configured with the load balancer’s public IP as its origin / backend.
- cert-manager in the cluster, with a DNS-01 capable
Issuer. In proxy mode HTTP-01 cannot reach Envoy (public DNS points at your proxy), so DNS-01 is used; it needs API credentials for your DNS provider. If you are not using our managed cert-manager, install your own.
Create the gateway namespace
kubectl create namespace gateway
Create the shared Gateway
Gateway describes the listeners. Put it in the dedicated gateway
namespace and reference the cluster GatewayClass named eg. The
allowedRoutes selector is what lets routes in other namespaces attach.
Create a file called gateway.yaml with the following content:
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: shared
namespace: gateway
spec:
gatewayClassName: eg
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
shared-gateway-access: "true"
- name: https
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
shared-gateway-access: "true"
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: team-a-tls
The Gateway has two listeners. The https listener (port 443) terminates TLS
for the re-encrypted hop from your upstream proxy; it has no hostname, so it
serves every host whose certificate is listed in its certificateRefs, and
Envoy picks the right one per request by SNI. The http listener (port 80) is
where an HTTP-to-HTTPS redirect would attach, if you run one here rather than at
your proxy (see below).
allowedRoutes.from: Selector admits routes from any namespace carrying the
shared-gateway-access: "true" label; this is the opt-in that makes the
Gateway shared. Use from: Same instead if you ever want a Gateway that only
serves its own namespace, or from: All to admit every namespace
unconditionally (not recommended, since it removes the opt-in).
Apply it:
kubectl apply -f gateway.yaml
Configure X-Forwarded-For with ClientTrafficPolicy
The ClientTrafficPolicy attaches to the Gateway by name and tells Envoy
how many trusted proxies sit in front of it. It lives in the gateway
namespace alongside the Gateway and covers the whole load balancer; app
namespaces do not need their own.
Create a file called client-traffic-policy.yaml:
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
name: shared
namespace: gateway
spec:
targetRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: shared
clientIPDetection:
xForwardedFor:
numTrustedHops: 1
numTrustedHops tells Envoy how many trusted ingress proxy hops sit in
front of it. Set it to the number of upstream proxies that prepend
entries to X-Forwarded-For. For a single CDN/WAF/edge proxy in front
of the load balancer, 1 is the right value; raise it for chains of
multiple proxies.
The setting affects what Envoy itself treats as the client IP, used in
access logs and rate limiting (and, in direct mode, the
x-envoy-external-address header).
Backends always see the full X-Forwarded-For chain that arrived
plus the load balancer’s internal IP appended on the right; Envoy does
not trim entries before forwarding the request upstream. Backend code
that needs the real client IP should parse the chain itself, typically
taking the leftmost public IP.
Note: The policy must live in the same namespace as the
Gateway. Envoy Gateway rejects cross-namespace policy targets.
Apply it:
kubectl apply -f client-traffic-policy.yaml
Alternative: using a custom header
Some load balancer setups forward the client IP in a different header.
Use customHeader instead; it is mutually exclusive with xForwardedFor:
clientIPDetection:
customHeader:
name: X-Real-IP
Issue a TLS certificate
Terminate TLS on the shared Gateway with a certificate per hostname, issued by
cert-manager into the gateway namespace (where TLS terminates). In proxy mode
your public DNS points at the upstream proxy rather than the load balancer, so an
HTTP-01 challenge would never reach Envoy. Use DNS-01 validation, which
proves control through a DNS record instead; it needs API credentials for your
DNS provider.
Create a file called certificate.yaml:
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-dns
namespace: gateway
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: platform@example.com
privateKeySecretRef:
name: letsencrypt-dns-account
solvers:
- dns01:
# Configure a DNS-01 solver for your DNS provider; see the
# cert-manager docs: https://cert-manager.io/docs/configuration/acme/dns01/
{}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: team-a-tls
namespace: gateway
spec:
secretName: team-a-tls
issuerRef:
name: letsencrypt-dns
kind: Issuer
dnsNames:
- team-a.example.com
Replace the email, DNS solver and hostname with your own, then apply it:
kubectl apply -f certificate.yaml
Each additional hostname needs its own Certificate and a matching entry in the
https listener’s certificateRefs; Envoy then selects the right certificate
per request by SNI.
Note: In proxy mode the upstream proxy terminates TLS for the public client; this certificate is for the re-encrypted hop between the upstream proxy and Envoy. Some setups instead let the proxy talk plain HTTP to the load balancer; if so, route via the
httplistener and you can skip the certificate.
Redirect HTTP to HTTPS
In this mode your upstream proxy (CDN / WAF / edge) usually performs the
HTTP-to-HTTPS redirect before traffic ever reaches the load balancer, so you
often do not need to configure one here. If you would rather have Envoy do it,
attach a redirect HTTPRoute to the http listener exactly as in the
direct-mode guide: label the gateway namespace
shared-gateway-access: "true" and apply a RequestRedirect route in it.
Onboard an application namespace
This is all an app team does; no access to the gateway namespace is needed.
1. Label the namespace so the shared Gateway admits its routes:
kubectl label namespace team-a shared-gateway-access=true
2. Deploy the app. Create app.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo
namespace: team-a
spec:
replicas: 1
selector:
matchLabels:
app: echo
template:
metadata:
labels:
app: echo
spec:
containers:
- name: echo
image: ealen/echo-server:0.9.2
ports:
- containerPort: 80
env:
- name: PORT
value: "80"
---
apiVersion: v1
kind: Service
metadata:
name: echo
namespace: team-a
spec:
selector:
app: echo
ports:
- port: 80
targetPort: 80
3. Route traffic to it. Create route.yaml. The parentRefs points at
the shared Gateway in the gateway namespace, and that cross-namespace
reference is what puts this app behind the shared load balancer:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: echo
namespace: team-a
spec:
parentRefs:
- name: shared
namespace: gateway
sectionName: https
hostnames:
- team-a.example.com
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: echo
port: 80
Apply both:
kubectl apply -f app.yaml -f route.yaml
The route’s hostname (team-a.example.com) must have a certificate on the
https listener; you issued one above. The backend Service is in the same
namespace as the HTTPRoute, so no ReferenceGrant is needed; you only need
one if a route points at a Service in a different namespace.
Verify
Check that the shared Gateway got an external address and that traffic
flows for the app namespace:
kubectl -n gateway get gateway shared -o jsonpath='{.status.addresses[0].value}'
curl -v https://team-a.example.com/
Confirm the route attached to the shared Gateway:
kubectl -n team-a get httproute echo -o jsonpath='{.status.parents[0].conditions}'
Accepted: True and ResolvedRefs: True mean the cross-namespace attach
worked. The backend should see the real client IP at the left of
X-Forwarded-For. Envoy forwards the full chain to the backend (including
the load balancer IP it appends on the right) and does not remove entries,
so the backend application is responsible for parsing the chain and picking
the leftmost public IP.
Common mistakes
- Namespace not labelled: the
HTTPRoutereportsAccepted: Falsewith reasonNotAllowedByListeners, and traffic never reaches the app. Label the app namespaceshared-gateway-access=true. - Forgetting
namespace:inparentRefs: without it the route looks for aGatewayin its own namespace, finds none, and stays unattached. Cross-namespace routes must name thegatewaynamespace. - No certificate for the hostname: if the
httpslistener has no certificate matching the route’s hostname, the TLS handshake fails and clients cannot connect. Issue aCertificatefor each hostname and add its Secret to the listener’scertificateRefs. - No upstream proxy in front: this variant assumes a CDN, WAF, or other proxy injects
X-Forwarded-Forbefore traffic reaches the load balancer. Without one, no real client IP arrives, and your backend will only see the LB’s internal IP. If you have no upstream proxy, use direct (PROXY-protocol) mode instead. - Forgetting
ClientTrafficPolicy: Envoy ignores the incomingX-Forwarded-Forand treats the load balancer’s internal IP as the client. Rate limiting and access logs see the LB, not your real client. - Putting
ClientTrafficPolicyin another namespace: silently ignored. Must be colocated with theGateway(here, thegatewaynamespace). - Wrong
numTrustedHops: too low and a caller can spoof the client IP by adding their ownX-Forwarded-Forentry. Too high and Envoy walks too far back into spoofable territory. Count one per trusted upstream proxy. - Mixing
xForwardedForandcustomHeader: they are mutually exclusive. Pick one.
Advanced usage
For more advanced use cases please refer to the documentation provided by each project or contact our support: