This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Envoy Gateway

Overview of the Envoy Gateway ingress in Elastx Kubernetes CaaS

This section introduces Envoy Gateway as the ingress controller in our Elastx Kubernetes CaaS service. We manage and upgrade the controller, the Gateway API CRDs and the cluster-scoped GatewayClass named eg. You create the Gateway API objects that describe your own traffic in your own namespaces.

There are companion guides for the two ways traffic typically reaches the cluster. Pick the one that matches your setup:

Standard layout: one shared Gateway per cluster

A cluster has one shared Gateway in a dedicated namespace (for example gateway) that serves routes from all your application namespaces through a single OpenStack load balancer and IP. This is the standard setup and the direct equivalent of ingress-nginx, where a single controller fronted every host in the cluster. Each application namespace opts in with a shared-gateway-access: "true" label and contributes its own HTTPRoutes, and never touches the shared Gateway.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#DAE7EC','primaryBorderColor':'#1E343E','primaryTextColor':'#1E343E','lineColor':'#5A7A8A','clusterBkg':'#EEF3F6','clusterBorder':'#9BB3BF','edgeLabelBackground':'#FFFFFF'}}}%%
flowchart TB
    client(["Clients"]):::client --> lb["OpenStack load balancer<br/>one LB &middot; one IP"]:::lb
    lb --> gw

    subgraph gwns["gateway namespace"]
        gw["Gateway 'shared'"]:::gw
        ctp["ClientTrafficPolicy"]:::policy
        cert["TLS certificates<br/>per hostname"]:::policy
    end
    ctp -.->|attaches to| gw
    cert -.->|terminates TLS| gw

    subgraph ta["team-a namespace"]
        appa["app + HTTPRoute"]:::app
    end
    subgraph tb["team-b namespace"]
        appb["app + HTTPRoute"]:::app
    end
    gw -->|"team-a.example.com"| appa
    gw -->|"team-b.example.com"| appb

    classDef client fill:#FFFFFF,stroke:#1E343E,color:#1E343E;
    classDef lb fill:#DAE7EC,stroke:#1E343E,color:#1E343E;
    classDef gw fill:#FBBD18,stroke:#1E343E,stroke-width:2px,color:#1E343E;
    classDef policy fill:#F5F8FA,stroke:#1E343E,color:#1E343E;
    classDef app fill:#DAE7EC,stroke:#1E343E,color:#1E343E;

Each Gateway provisions its own OpenStack load balancer, so a single shared Gateway keeps your cluster on one load balancer, one IP and one ClientTrafficPolicy: the same single-entry-point model you had with ingress-nginx. TLS is terminated centrally in the gateway namespace, with one certificate per hostname (Envoy serves the right one per request by SNI). The walkthroughs below use this layout throughout. Running more than one Gateway, a separate load balancer for a single namespace via allowedRoutes.namespaces.from: Same, is a non-standard setup for the rare case that genuinely needs an isolated IP or blast radius.

What you create

In the dedicated gateway namespace (once, by whoever owns ingress):

  • Gateway: listeners, ports, protocols, TLS; allowedRoutes selecting the shared-gateway-access: "true" label.
  • ClientTrafficPolicy: controls PROXY-protocol handling, TLS parameters, timeouts. Must live in the same namespace as the Gateway.
  • TLS Certificate / Issuer (cert-manager): one certificate per hostname you serve (HTTP-01 in direct mode, DNS-01 in proxy mode).
  • Optionally, an HTTP-to-HTTPS redirect HTTPRoute on the http listener, to match ingress-nginx’s ssl-redirect behaviour for every host.

In each application namespace (per team, self-service):

  • The shared-gateway-access: "true" namespace label.
  • HTTPRoute, GRPCRoute: routing rules, attached to the shared Gateway via cross-namespace parentRefs.
  • BackendTrafficPolicy: retries, circuit breaking.
  • SecurityPolicy: JWT, OIDC, CORS.
  • BackendTLSPolicy: mTLS toward your backends.

You reference the cluster GatewayClass by its name eg from the Gateway. You do not need to create or modify any cluster-scoped resources.

Which variant fits your setup?

The OpenStack load balancer in front of Envoy runs in TCP mode in both cases. The variants differ in how the real client IP arrives at Envoy, and your ClientTrafficPolicy has to match.

  • Direct (PROXY-protocol) mode: clients connect straight to the load balancer. The load balancer is configured with PROXY protocol v2 and prepends a PROXY header carrying the real client IP. Your ClientTrafficPolicy must enable proxy-protocol parsing. See Direct (PROXY-protocol) mode.
  • Proxy (X-Forwarded-For) mode: you put your own upstream proxy (CDN, WAF, edge proxy) in front of the load balancer. That upstream injects the real client IP into X-Forwarded-For; the load balancer passes the request through unchanged. Your ClientTrafficPolicy must trust that header with the right hop count. See Proxy (X-Forwarded-For) mode.

Coming from ingress-nginx?

If you used our managed ingress-nginx, the two modes carry over directly; only the names and the resources you write have changed:

ingress-nginx Envoy Gateway When it applies
Direct mode (use-proxy-protocol: "true") Direct (PROXY-protocol) mode Clients connect straight to our load balancer. No upstream proxy. This is the default.
Proxy mode (use-forwarded-headers: "true") Proxy (X-Forwarded-For) mode You run your own CDN / WAF / edge proxy in front of the load balancer.

As before, the mode is a cluster-level setting: we provision your cluster in one mode or the other; you do not switch it from a manifest. Tell us which fits your setup and we configure the load balancer accordingly. What you do write is a ClientTrafficPolicy that matches that mode (see the two guides above).

Proxy mode requires your own upstream proxy. X-Forwarded-For mode only makes sense when a CDN, WAF, or edge proxy actually sits in front of the load balancer and injects the header. Without one, no real client IP ever reaches Envoy and your backends see only the load balancer. If clients connect directly to Elastx, use direct (PROXY-protocol) mode instead. It carries the client IP for you and needs no CDN.

Two things also changed with Kubernetes CaaS v2:

  • Ingress always enters through the load balancer. Older clusters accepted traffic directly on each worker node’s floating IP; that path is gone. DNS for your services now points at the Gateway’s load-balancer address.
  • Floating IPs are an opt-in egress feature, used for a predictable outbound source IP, not as an ingress path.

For the full move, including clusters that use floating IPs, see Migrating from ingress-nginx.

TLS

Both walkthroughs terminate TLS on the shared Gateway, with cert-manager issuing one certificate per hostname into the gateway namespace (the same namespace as the Gateway). The validation method differs by mode: direct mode uses ACME HTTP-01 (clients reach the load balancer directly), while proxy mode uses DNS-01 (public DNS points at your upstream proxy, so HTTP-01 cannot reach Envoy). If you need a guide for installing cert-manager, see Install and upgrade cert-manager.

Advanced usage

For more advanced use cases please refer to the documentation provided by each project or contact our support:

1 - Direct (PROXY-protocol) mode

A walkthrough of setting up Envoy Gateway when your cluster’s load balancer uses PROXY protocol v2

This guide walks through setting up Envoy Gateway in a cluster where the OpenStack load balancer is configured in TCP mode with PROXY protocol v2. The load balancer prepends a PROXY header to each incoming connection carrying the real client IP. Envoy parses that header and uses it for access logs, rate limiting and X-Forwarded-For.

Note: Your ClientTrafficPolicy must set proxyProtocol.optional: false. Without it Envoy parses the load balancer’s PROXY-v2 prefix as a malformed HTTP request and every response is HTTP 400 Bad Request.

If you are not sure which variant applies to your cluster, see the Envoy Gateway overview.

The shared-Gateway layout

A cluster runs one shared Gateway in a dedicated namespace that serves routes from all your application namespaces through a single load balancer and IP. Each application namespace opts in with a label and contributes its own HTTPRoutes. This is the standard setup, the same single-entry-point model ingress-nginx gave you, where one controller fronted every host.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#DAE7EC','primaryBorderColor':'#1E343E','primaryTextColor':'#1E343E','lineColor':'#5A7A8A','clusterBkg':'#EEF3F6','clusterBorder':'#9BB3BF','edgeLabelBackground':'#FFFFFF'}}}%%
flowchart TB
    client(["Clients"]):::client -->|PROXY protocol v2| lb["OpenStack load balancer<br/>TCP mode &middot; one LB &middot; one IP"]:::lb
    lb --> gw

    subgraph gwns["gateway namespace"]
        gw["Gateway 'shared'"]:::gw
        ctp["ClientTrafficPolicy<br/>proxyProtocol.optional: false"]:::policy
        cert["TLS certificate<br/>team-a.example.com"]:::policy
    end
    ctp -.->|attaches to| gw
    cert -.->|terminates TLS| gw

    subgraph ta["team-a namespace (labelled)"]
        appa["app + HTTPRoute"]:::app
    end
    subgraph tb["team-b namespace (labelled)"]
        appb["app + HTTPRoute"]:::app
    end
    gw -->|"team-a.example.com"| appa
    gw -->|"team-b.example.com"| appb

    classDef client fill:#FFFFFF,stroke:#1E343E,color:#1E343E;
    classDef lb fill:#DAE7EC,stroke:#1E343E,color:#1E343E;
    classDef gw fill:#FBBD18,stroke:#1E343E,stroke-width:2px,color:#1E343E;
    classDef policy fill:#F5F8FA,stroke:#1E343E,color:#1E343E;
    classDef app fill:#DAE7EC,stroke:#1E343E,color:#1E343E;

What the shared Gateway gives you:

  • One load balancer per cluster: a single LB and IP front all your teams, the way a single ingress controller did before.
  • One ClientTrafficPolicy to manage, in the gateway namespace, with TLS terminated there (one certificate per hostname you serve).
  • Self-service for app teams: an app team only labels its namespace and creates an HTTPRoute; it never touches the shared Gateway.

Each Gateway provisions its own load balancer. Running more than one (a separate Gateway and load balancer for a single namespace, via allowedRoutes.namespaces.from: Same) is a non-standard setup, for the rare case that genuinely needs an isolated IP or blast radius.

Prerequisites

  • A dedicated namespace for the shared Gateway. The examples use gateway.
  • One or more application namespaces. The examples use team-a.
  • A DNS record for each hostname you serve, pointing at the load balancer’s public IP. The examples use team-a.example.com; replace it with your own throughout.
  • cert-manager in the cluster. The examples issue certificates with ACME HTTP-01, which needs no DNS-provider credentials. If you are not using our managed cert-manager, install your own.

Create the gateway namespace

kubectl create namespace gateway
kubectl label namespace gateway shared-gateway-access=true

The label lets routes created in the gateway namespace itself attach to the shared Gateway. Two gateway-owner routes need this: cert-manager’s short-lived HTTP-01 challenge route, and the HTTP-to-HTTPS redirect below.

Create the shared Gateway

Gateway describes the listeners. Put it in the dedicated gateway namespace and reference the cluster GatewayClass named eg. The allowedRoutes selector is what lets routes in other namespaces attach.

Create a file called gateway.yaml with the following content:

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared
  namespace: gateway
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"
    - name: https
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: team-a-tls

The Gateway has two listeners. The https listener (port 443) terminates TLS; it has no hostname, so it serves every host whose certificate is listed in its certificateRefs, and Envoy picks the right one per request by SNI. The http listener (port 80) carries plaintext requests: it serves ACME HTTP-01 challenges and is where the HTTP-to-HTTPS redirect below attaches.

allowedRoutes.from: Selector admits routes from any namespace carrying the shared-gateway-access: "true" label; this is the opt-in that makes the Gateway shared. Use from: Same instead if you ever want a Gateway that only serves its own namespace, or from: All to admit every namespace unconditionally (not recommended, since it removes the opt-in).

Apply it: kubectl apply -f gateway.yaml

Configure proxy-protocol with ClientTrafficPolicy

The ClientTrafficPolicy attaches to the Gateway by name and tells Envoy to parse the PROXY-v2 header from the load balancer. It lives in the gateway namespace alongside the Gateway and covers the whole load balancer; app namespaces do not need their own.

Create a file called client-traffic-policy.yaml:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: shared
  namespace: gateway
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: shared
  proxyProtocol:
    optional: false

Note: The policy must live in the same namespace as the Gateway. Envoy Gateway rejects cross-namespace policy targets.

Apply it: kubectl apply -f client-traffic-policy.yaml

Issue a TLS certificate

Terminate TLS on the shared Gateway with a certificate per hostname, issued by cert-manager into the gateway namespace (where TLS terminates). In direct mode clients reach the load balancer directly, so ACME HTTP-01 is the simplest validation: Let’s Encrypt fetches a token over port 80, which the shared Gateway already serves, and no DNS-provider credentials are needed.

Create a file called certificate.yaml:

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-http01
  namespace: gateway
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform@example.com
    privateKeySecretRef:
      name: letsencrypt-http01-account
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - group: gateway.networking.k8s.io
                kind: Gateway
                name: shared
                namespace: gateway
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: team-a-tls
  namespace: gateway
spec:
  secretName: team-a-tls
  issuerRef:
    name: letsencrypt-http01
    kind: Issuer
  dnsNames:
    - team-a.example.com

Replace the email and hostname with your own, then apply it: kubectl apply -f certificate.yaml

To solve the challenge, cert-manager creates a short-lived HTTPRoute in the gateway namespace, attached to the http listener. It is admitted because you labelled the gateway namespace when you created it. The https listener stays pending until the certificate is issued, then serves it.

Each additional hostname needs its own Certificate and a matching entry in the https listener’s certificateRefs; Envoy then selects the right certificate per request by SNI.

Prefer DNS-01? If you would rather validate over DNS (for example to keep the ACME servers off port 80), use a DNS-01 Issuer instead. DNS-01 needs API credentials for your DNS provider; see the cert-manager DNS-01 docs.

Redirect HTTP to HTTPS

ingress-nginx redirected HTTP to HTTPS for you (the ssl-redirect default). To keep that behaviour, attach one redirect HTTPRoute to the http listener, created once in the gateway namespace (already labelled above). It matches every host on port 80, so app teams do not add their own.

Create https-redirect.yaml:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: https-redirect
  namespace: gateway
spec:
  parentRefs:
    - name: shared
      namespace: gateway
      sectionName: http
  rules:
    - filters:
        - type: RequestRedirect
          requestRedirect:
            scheme: https
            statusCode: 301

Apply it: kubectl apply -f https-redirect.yaml

A request to http://team-a.example.com/ now returns 301 Moved Permanently with Location: https://team-a.example.com/, and the client repeats the request over HTTPS.

Does this break HTTP-01 certificates? No. This redirect matches the path /, while cert-manager’s challenge route matches the longer /.well-known/acme-challenge/ path. Gateway API gives precedence to the longest path match, so ACME challenges are still served over plain HTTP while everything else redirects.

Onboard an application namespace

This is all an app team does; no access to the gateway namespace is needed.

1. Label the namespace so the shared Gateway admits its routes:

kubectl label namespace team-a shared-gateway-access=true

2. Deploy the app. Create app.yaml:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo
  namespace: team-a
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
        - name: echo
          image: ealen/echo-server:0.9.2
          ports:
            - containerPort: 80
          env:
            - name: PORT
              value: "80"
---
apiVersion: v1
kind: Service
metadata:
  name: echo
  namespace: team-a
spec:
  selector:
    app: echo
  ports:
    - port: 80
      targetPort: 80

3. Route traffic to it. Create route.yaml. The parentRefs points at the shared Gateway in the gateway namespace, and that cross-namespace reference is what puts this app behind the shared load balancer:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
  namespace: team-a
spec:
  parentRefs:
    - name: shared
      namespace: gateway
      sectionName: https
  hostnames:
    - team-a.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: echo
          port: 80

Apply both: kubectl apply -f app.yaml -f route.yaml

The route’s hostname (team-a.example.com) must have a certificate on the https listener; you issued one above. The backend Service is in the same namespace as the HTTPRoute, so no ReferenceGrant is needed; you only need one if a route points at a Service in a different namespace.

Verify

Check that the shared Gateway got an external address and that traffic flows for the app namespace:

kubectl -n gateway get gateway shared -o jsonpath='{.status.addresses[0].value}'
curl -v https://team-a.example.com/

Confirm the route attached to the shared Gateway:

kubectl -n team-a get httproute echo -o jsonpath='{.status.parents[0].conditions}'

Accepted: True and ResolvedRefs: True mean the cross-namespace attach worked. The backend should see the real client IP in X-Forwarded-For and X-Envoy-External-Address.

Common mistakes

  • Namespace not labelled: the HTTPRoute reports Accepted: False with reason NotAllowedByListeners, and traffic never reaches the app. Label the app namespace shared-gateway-access=true.
  • Forgetting namespace: in parentRefs: without it the route looks for a Gateway in its own namespace, finds none, and stays unattached. Cross-namespace routes must name the gateway namespace.
  • No certificate for the hostname: if the https listener has no certificate matching the route’s hostname, the TLS handshake fails and clients cannot connect. Issue a Certificate for each hostname and add its Secret to the listener’s certificateRefs.
  • Forgetting ClientTrafficPolicy: every request returns HTTP 400 Bad Request. The load balancer is prepending a PROXY-v2 binary header; without the policy Envoy treats those bytes as the start of an HTTP request and fails to parse it.
  • Putting ClientTrafficPolicy in another namespace: silently ignored. Must be colocated with the Gateway (here, the gateway namespace).
  • Setting proxyProtocol.optional: true: opens you up to clients that don’t send the header bypassing client-IP enforcement. Keep it false.
  • Testing with curl from outside the load balancer: PROXY-protocol traffic isn’t valid HTTP. Always go through the load balancer’s VIP.

Advanced usage

For more advanced use cases please refer to the documentation provided by each project or contact our support:

2 - Proxy (X-Forwarded-For) mode

A walkthrough of setting up Envoy Gateway when your traffic arrives via an upstream proxy that injects X-Forwarded-For

This guide walks through setting up Envoy Gateway in a cluster where you front the OpenStack load balancer with your own upstream proxy (for example a CDN, WAF, or edge proxy) that terminates the client connection and injects the real client IP into the X-Forwarded-For header. The OpenStack load balancer itself stays in TCP passthrough; the upstream proxy is what carries the client IP for you.

This mode requires your own upstream proxy. It is only correct when a CDN, WAF, or edge proxy that you operate sits in front of the load balancer and injects X-Forwarded-For. Without one, no real client IP ever reaches Envoy, so your backends see only the load balancer’s internal IP. If clients connect straight to Elastx with nothing in front, use direct (PROXY-protocol) mode instead; it carries the client IP for you and needs no proxy.

Note: Your ClientTrafficPolicy must set clientIPDetection.xForwardedFor with numTrustedHops set to the number of trusted proxies in front of Envoy. Without it Envoy will not honour the incoming X-Forwarded-For header and your access logs and rate limiting will see the load balancer’s internal IP.

Note: The upstream proxy must terminate TLS to inject X-Forwarded-For; it can only read and modify headers on decrypted traffic. It then opens a fresh connection to the load balancer (which passes it through untouched) and Envoy terminates TLS again on the Gateway HTTPS listener. A pure TCP/TLS passthrough upstream cannot inject the header.

If you are not sure which variant applies to your cluster, see the Envoy Gateway overview.

The shared-Gateway layout

A cluster runs one shared Gateway in a dedicated namespace that serves routes from all your application namespaces through a single load balancer and IP. Each application namespace opts in with a label and contributes its own HTTPRoutes. This is the standard setup, the same single-entry-point model ingress-nginx gave you, where one controller fronted every host.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#DAE7EC','primaryBorderColor':'#1E343E','primaryTextColor':'#1E343E','lineColor':'#5A7A8A','clusterBkg':'#EEF3F6','clusterBorder':'#9BB3BF','edgeLabelBackground':'#FFFFFF'}}}%%
flowchart TB
    client(["Clients"]):::client --> proxy["Your upstream proxy<br/>CDN / WAF / edge<br/>injects X-Forwarded-For"]:::proxy
    proxy -->|TCP passthrough| lb["OpenStack load balancer<br/>TCP mode &middot; one LB &middot; one IP"]:::lb
    lb --> gw

    subgraph gwns["gateway namespace"]
        gw["Gateway 'shared'"]:::gw
        ctp["ClientTrafficPolicy<br/>xForwardedFor.numTrustedHops: 1"]:::policy
        cert["TLS certificate<br/>team-a.example.com"]:::policy
    end
    ctp -.->|attaches to| gw
    cert -.->|terminates TLS| gw

    subgraph ta["team-a namespace (labelled)"]
        appa["app + HTTPRoute"]:::app
    end
    subgraph tb["team-b namespace (labelled)"]
        appb["app + HTTPRoute"]:::app
    end
    gw -->|"team-a.example.com"| appa
    gw -->|"team-b.example.com"| appb

    classDef client fill:#FFFFFF,stroke:#1E343E,color:#1E343E;
    classDef proxy fill:#CFE8FF,stroke:#0041C2,color:#1E343E;
    classDef lb fill:#DAE7EC,stroke:#1E343E,color:#1E343E;
    classDef gw fill:#FBBD18,stroke:#1E343E,stroke-width:2px,color:#1E343E;
    classDef policy fill:#F5F8FA,stroke:#1E343E,color:#1E343E;
    classDef app fill:#DAE7EC,stroke:#1E343E,color:#1E343E;

What the shared Gateway gives you:

  • One load balancer per cluster: a single LB and IP front all your teams, the way a single ingress controller did before, and your upstream proxy points at a single origin IP.
  • One ClientTrafficPolicy to manage, in the gateway namespace, with TLS terminated there (one certificate per hostname you serve).
  • Self-service for app teams: an app team only labels its namespace and creates an HTTPRoute; it never touches the shared Gateway.

Each Gateway provisions its own load balancer. Running more than one (a separate Gateway and load balancer for a single namespace, via allowedRoutes.namespaces.from: Same) is a non-standard setup, for the rare case that genuinely needs an isolated IP or blast radius.

Prerequisites

  • A dedicated namespace for the shared Gateway. The examples use gateway.
  • One or more application namespaces. The examples use team-a.
  • DNS for each public hostname (the examples use team-a.example.com, replace with your own) pointing at your upstream proxy (CDN / WAF / edge proxy), not directly at the load balancer. If it resolves straight to the load balancer, traffic bypasses the proxy and no X-Forwarded-For is injected.
  • Your upstream proxy configured with the load balancer’s public IP as its origin / backend.
  • cert-manager in the cluster, with a DNS-01 capable Issuer. In proxy mode HTTP-01 cannot reach Envoy (public DNS points at your proxy), so DNS-01 is used; it needs API credentials for your DNS provider. If you are not using our managed cert-manager, install your own.

Create the gateway namespace

kubectl create namespace gateway

Create the shared Gateway

Gateway describes the listeners. Put it in the dedicated gateway namespace and reference the cluster GatewayClass named eg. The allowedRoutes selector is what lets routes in other namespaces attach.

Create a file called gateway.yaml with the following content:

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared
  namespace: gateway
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"
    - name: https
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: team-a-tls

The Gateway has two listeners. The https listener (port 443) terminates TLS for the re-encrypted hop from your upstream proxy; it has no hostname, so it serves every host whose certificate is listed in its certificateRefs, and Envoy picks the right one per request by SNI. The http listener (port 80) is where an HTTP-to-HTTPS redirect would attach, if you run one here rather than at your proxy (see below).

allowedRoutes.from: Selector admits routes from any namespace carrying the shared-gateway-access: "true" label; this is the opt-in that makes the Gateway shared. Use from: Same instead if you ever want a Gateway that only serves its own namespace, or from: All to admit every namespace unconditionally (not recommended, since it removes the opt-in).

Apply it: kubectl apply -f gateway.yaml

Configure X-Forwarded-For with ClientTrafficPolicy

The ClientTrafficPolicy attaches to the Gateway by name and tells Envoy how many trusted proxies sit in front of it. It lives in the gateway namespace alongside the Gateway and covers the whole load balancer; app namespaces do not need their own.

Create a file called client-traffic-policy.yaml:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: shared
  namespace: gateway
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: shared
  clientIPDetection:
    xForwardedFor:
      numTrustedHops: 1

numTrustedHops tells Envoy how many trusted ingress proxy hops sit in front of it. Set it to the number of upstream proxies that prepend entries to X-Forwarded-For. For a single CDN/WAF/edge proxy in front of the load balancer, 1 is the right value; raise it for chains of multiple proxies.

The setting affects what Envoy itself treats as the client IP, used in access logs and rate limiting (and, in direct mode, the x-envoy-external-address header). Backends always see the full X-Forwarded-For chain that arrived plus the load balancer’s internal IP appended on the right; Envoy does not trim entries before forwarding the request upstream. Backend code that needs the real client IP should parse the chain itself, typically taking the leftmost public IP.

Note: The policy must live in the same namespace as the Gateway. Envoy Gateway rejects cross-namespace policy targets.

Apply it: kubectl apply -f client-traffic-policy.yaml

Alternative: using a custom header

Some load balancer setups forward the client IP in a different header. Use customHeader instead; it is mutually exclusive with xForwardedFor:

  clientIPDetection:
    customHeader:
      name: X-Real-IP

Issue a TLS certificate

Terminate TLS on the shared Gateway with a certificate per hostname, issued by cert-manager into the gateway namespace (where TLS terminates). In proxy mode your public DNS points at the upstream proxy rather than the load balancer, so an HTTP-01 challenge would never reach Envoy. Use DNS-01 validation, which proves control through a DNS record instead; it needs API credentials for your DNS provider.

Create a file called certificate.yaml:

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-dns
  namespace: gateway
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: platform@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-account
    solvers:
      - dns01:
          # Configure a DNS-01 solver for your DNS provider; see the
          # cert-manager docs: https://cert-manager.io/docs/configuration/acme/dns01/
          {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: team-a-tls
  namespace: gateway
spec:
  secretName: team-a-tls
  issuerRef:
    name: letsencrypt-dns
    kind: Issuer
  dnsNames:
    - team-a.example.com

Replace the email, DNS solver and hostname with your own, then apply it: kubectl apply -f certificate.yaml

Each additional hostname needs its own Certificate and a matching entry in the https listener’s certificateRefs; Envoy then selects the right certificate per request by SNI.

Note: In proxy mode the upstream proxy terminates TLS for the public client; this certificate is for the re-encrypted hop between the upstream proxy and Envoy. Some setups instead let the proxy talk plain HTTP to the load balancer; if so, route via the http listener and you can skip the certificate.

Redirect HTTP to HTTPS

In this mode your upstream proxy (CDN / WAF / edge) usually performs the HTTP-to-HTTPS redirect before traffic ever reaches the load balancer, so you often do not need to configure one here. If you would rather have Envoy do it, attach a redirect HTTPRoute to the http listener exactly as in the direct-mode guide: label the gateway namespace shared-gateway-access: "true" and apply a RequestRedirect route in it.

Onboard an application namespace

This is all an app team does; no access to the gateway namespace is needed.

1. Label the namespace so the shared Gateway admits its routes:

kubectl label namespace team-a shared-gateway-access=true

2. Deploy the app. Create app.yaml:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo
  namespace: team-a
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
        - name: echo
          image: ealen/echo-server:0.9.2
          ports:
            - containerPort: 80
          env:
            - name: PORT
              value: "80"
---
apiVersion: v1
kind: Service
metadata:
  name: echo
  namespace: team-a
spec:
  selector:
    app: echo
  ports:
    - port: 80
      targetPort: 80

3. Route traffic to it. Create route.yaml. The parentRefs points at the shared Gateway in the gateway namespace, and that cross-namespace reference is what puts this app behind the shared load balancer:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
  namespace: team-a
spec:
  parentRefs:
    - name: shared
      namespace: gateway
      sectionName: https
  hostnames:
    - team-a.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: echo
          port: 80

Apply both: kubectl apply -f app.yaml -f route.yaml

The route’s hostname (team-a.example.com) must have a certificate on the https listener; you issued one above. The backend Service is in the same namespace as the HTTPRoute, so no ReferenceGrant is needed; you only need one if a route points at a Service in a different namespace.

Verify

Check that the shared Gateway got an external address and that traffic flows for the app namespace:

kubectl -n gateway get gateway shared -o jsonpath='{.status.addresses[0].value}'
curl -v https://team-a.example.com/

Confirm the route attached to the shared Gateway:

kubectl -n team-a get httproute echo -o jsonpath='{.status.parents[0].conditions}'

Accepted: True and ResolvedRefs: True mean the cross-namespace attach worked. The backend should see the real client IP at the left of X-Forwarded-For. Envoy forwards the full chain to the backend (including the load balancer IP it appends on the right) and does not remove entries, so the backend application is responsible for parsing the chain and picking the leftmost public IP.

Common mistakes

  • Namespace not labelled: the HTTPRoute reports Accepted: False with reason NotAllowedByListeners, and traffic never reaches the app. Label the app namespace shared-gateway-access=true.
  • Forgetting namespace: in parentRefs: without it the route looks for a Gateway in its own namespace, finds none, and stays unattached. Cross-namespace routes must name the gateway namespace.
  • No certificate for the hostname: if the https listener has no certificate matching the route’s hostname, the TLS handshake fails and clients cannot connect. Issue a Certificate for each hostname and add its Secret to the listener’s certificateRefs.
  • No upstream proxy in front: this variant assumes a CDN, WAF, or other proxy injects X-Forwarded-For before traffic reaches the load balancer. Without one, no real client IP arrives, and your backend will only see the LB’s internal IP. If you have no upstream proxy, use direct (PROXY-protocol) mode instead.
  • Forgetting ClientTrafficPolicy: Envoy ignores the incoming X-Forwarded-For and treats the load balancer’s internal IP as the client. Rate limiting and access logs see the LB, not your real client.
  • Putting ClientTrafficPolicy in another namespace: silently ignored. Must be colocated with the Gateway (here, the gateway namespace).
  • Wrong numTrustedHops: too low and a caller can spoof the client IP by adding their own X-Forwarded-For entry. Too high and Envoy walks too far back into spoofable territory. Count one per trusted upstream proxy.
  • Mixing xForwardedFor and customHeader: they are mutually exclusive. Pick one.

Advanced usage

For more advanced use cases please refer to the documentation provided by each project or contact our support:

3 - Migrating from ingress-nginx

How to move from managed ingress-nginx to Envoy Gateway, with or without floating IPs and in direct or proxy mode

This guide helps you move an existing workload from our managed ingress-nginx to Envoy Gateway. The ingress concepts are the same (listeners, routes, TLS, and a choice between direct and proxy mode), but the resources you write are Gateway API objects instead of Ingress objects.

If you are not migrating but setting up fresh, start from the Envoy Gateway overview instead.

What changed

  • Ingress always enters through the load balancer. ingress-nginx on older clusters could accept traffic directly on each worker node (often on a node floating IP). With Envoy Gateway, traffic always arrives through a single OpenStack load balancer that fronts the Envoy data plane. Your public DNS points at that load balancer, not at nodes.
  • Floating IPs are now an egress feature. In Kubernetes CaaS v2 floating IPs are removed from nodes by default and are an opt-in feature whose purpose is a predictable outbound source IP. They are not part of the ingress path. See Floating IPs below.
  • Direct vs proxy is still a cluster-level mode, set by us: the same choice you made with ingress-nginx (use-proxy-protocol vs use-forwarded-headers), just under new names.

Step 0: Confirm your mode (direct or proxy)

The mode must match how your cluster’s load balancer is provisioned. It is a cluster-level setting that we manage; if you are unsure which one your cluster runs, ask support before you cut over.

  • Direct (PROXY-protocol) mode: clients connect straight to our load balancer. The load balancer carries the real client IP with PROXY protocol v2. No CDN or upstream proxy is involved. This is the default and the equivalent of ingress-nginx use-proxy-protocol: "true".
  • Proxy (X-Forwarded-For) mode: you operate your own CDN / WAF / edge proxy in front of the load balancer, and it injects X-Forwarded-For. The equivalent of ingress-nginx use-forwarded-headers: "true".

Proxy mode requires your own upstream proxy. It only makes sense if a CDN, WAF, or edge proxy actually sits in front of the load balancer and injects the header. Without one, no real client IP reaches Envoy and your backends see only the load balancer. If clients connect directly to Elastx, use direct (PROXY-protocol) mode; it needs no CDN.

Once you know your mode, the per-resource walkthrough lives in:

Resource mapping

ingress-nginx Envoy Gateway / Gateway API
IngressClass nginx GatewayClass eg (cluster-managed; you only reference it)
Ingress (one object, implicit listeners) Gateway (explicit listeners, ports, TLS) + HTTPRoute (routing rules)
spec.tls on the Ingress HTTPS listener tls.certificateRefs on the Gateway
use-proxy-protocol / use-forwarded-headers (controller ConfigMap) Cluster mode (Elastx) + your ClientTrafficPolicy
nginx.ingress.kubernetes.io/* annotations HTTPRoute filters, BackendTrafficPolicy, SecurityPolicy
cert-manager Issuer with solvers.http01.ingress.class: nginx cert-manager Issuer with solvers.http01.gatewayHTTPRoute.parentRefs
TCP/UDP services (tcp-services ConfigMap) TCPRoute / UDPRoute (see note below)

Note: Our Envoy Gateway ships the Gateway API standard channel, which provides only HTTPRoute and GRPCRoute. TCPRoute, TLSRoute and UDPRoute are not installed. If you relied on the nginx tcp-services ConfigMap, contact support before migrating those.

Migration steps

  1. Confirm your mode with Elastx (direct or proxy; see Step 0).
  2. Make sure cert-manager is available. If you are not using our managed cert-manager, install and configure your own (with an Issuer); see Install and upgrade cert-manager.
  3. Recreate your ingress as Gateway API objects following the guide for your mode (direct or proxy). The standard layout is a shared Gateway in a dedicated gateway namespace, with a matching ClientTrafficPolicy and a TLS Certificate per hostname, that your application namespaces attach to. Each app namespace then only carries the shared-gateway-access: "true" label and one HTTPRoute per host. A single ingress-nginx controller served all your hosts through one entry point before; one shared Gateway is its direct equivalent, keeping your cluster on a single load balancer.
  4. Translate annotations. Path rewrites, header manipulation and redirects become HTTPRoute filters; retries and circuit breaking become a BackendTrafficPolicy; auth/CORS become a SecurityPolicy.
  5. Switch your cert-manager Issuer to a Gateway-aware solver that issues a certificate per hostname: the http01.gatewayHTTPRoute solver in direct mode, or a DNS-01 Issuer in proxy mode (where public DNS points at your upstream proxy). Each mode guide shows the one for that mode.
  6. Test before cutover without touching DNS, by resolving your hostname to the new Gateway address locally:
    GW=$(kubectl -n gateway get gateway shared -o jsonpath='{.status.addresses[0].value}')
    curl -v --resolve your.host.example.com:443:"$GW" https://your.host.example.com/
    
  7. Cut over DNS to the Gateway’s load-balancer address. Use a low TTL (≈1 minute) beforehand so the change propagates quickly, and a CNAME where possible so only one record needs updating.
  8. Decommission ingress-nginx once traffic is confirmed on Envoy Gateway. Avoid running two controllers on the same IngressClass during the overlap.

Floating IPs

How you migrate depends on what your floating IPs were doing:

  • You used node floating IPs as your ingress entry point (older clusters). That path no longer exists; ingress now enters through the load balancer. Point your DNS at the Gateway’s load-balancer address (Step 7). Nothing about Envoy Gateway changes between “had node FIPs” and “did not”; the entry point is the load balancer either way.
  • You need a predictable egress (outbound) source IP. Floating IPs are still available for that, as an opt-in feature; without them, egress is SNAT’ed via the hypervisor. This is independent of ingress and of the direct/proxy choice. If you want to keep or enable floating IPs, let support know; toggling them recreates your nodes.

Advanced usage

For more advanced use cases please refer to the documentation provided by each project or contact our support: