Skip to content

Commit

Permalink
docs: Traefik guide updated to cover Ingress, IngressRoute and `H…
Browse files Browse the repository at this point in the history
…TTPRoute` based integration options (#1420)
  • Loading branch information
dadrus authored May 17, 2024
1 parent 55fb491 commit 303095e
Show file tree
Hide file tree
Showing 21 changed files with 307 additions and 27 deletions.
4 changes: 2 additions & 2 deletions docs/content/guides/proxies/haproxy.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ data:
auth-url: "https://<heimdall service name>.<namespace>.svc.cluster.local:<decision port>" # <1>
auth-headers-succeed: "authorization" # <2>
headers: | # <3>
X-Forwarded-Uri: %[baseq]
X-Forwarded-Uri: %[pathq]
X-Forwarded-Method: %[method]
X-Forwarded-Host: %[req.hdr(host)]
----
Expand All @@ -63,7 +63,7 @@ annotations:
haproxy-ingress.github.io/auth-url: "https://<heimdall service name>.<namespace>.svc.cluster.local:<decision port>"
haproxy-ingress.github.io/auth-headers-succeed: "authorization"
haproxy-ingress.github.io/headers: |
X-Forwarded-Uri: %[baseq]
X-Forwarded-Uri: %[pathq]
X-Forwarded-Method: %[method]
X-Forwarded-Host: %[req.hdr(host)]
----
Expand Down
2 changes: 1 addition & 1 deletion docs/content/guides/proxies/nginx.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Even one can configure an external auth server globally with vanilla NGINX, ther

To integrate heimdall with the NGINX Ingress Controller you can make use of the `nginx.ingress.kubernetes.io/auth-url`, `nginx.ingress.kubernetes.io/auth-response-headers` and the `nginx.ingress.kubernetes.io/auth-snippet` annotation as shown in the example below. This will result in an NGINX configuration corresponding to the integration option, described in the link:{{< relref "#_second_option" >}}[Forward all information in `X-Forwarded-*` headers] section.

NOTE: The configuration used in the example below requires proper configuration of `trusted_proxies`.
NOTE: The configuration used in the example below requires proper configuration of `trusted_proxies` on heimdall side. On NGINX Ingress Controller side you must allow the usage of `nginx.ingress.kubernetes.io/auth-snippet` (See also https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#allow-snippet-annotations[here]).

[source, yaml]
----
Expand Down
88 changes: 84 additions & 4 deletions docs/content/guides/proxies/traefik.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ https://doc.traefik.io/traefik/[Traefik Proxy] is a modern HTTP proxy and load b

== Prerequisites

* Integration with Envoy proxy requires heimdall being operated in link:{{< relref "/docs/concepts/operating_modes.adoc#_decision_mode" >}}[Decision Operation Mode].
* Integration with traefik requires heimdall being operated in link:{{< relref "/docs/concepts/operating_modes.adoc#_decision_mode" >}}[Decision Operation Mode].

[CAUTION]
====
Expand All @@ -31,12 +31,14 @@ To let Traefik forward all incoming requests to heimdall, there is a need
* to configure the https://doc.traefik.io/traefik/middlewares/http/forwardauth/[ForwardAuth] middleware, and
* to add it to the list of https://doc.traefik.io/traefik/routing/entrypoints/#middlewares[middlewares] that are prepended by default to the list of middlewares of each router associated to a named entry point.

Both is shown in the snippet below
=== Regular Deployment

If you are using Traefik outside of kubernetes, the above can be achieved by the following static configuration

[source, yaml]
----
entryPoints:
http:
web:
address: ":8080"
middlewares: # <1>
- heimdall
Expand All @@ -56,6 +58,51 @@ http:
<4> Configures this middleware to forward requests to a service available under "heimdall" DNS name
<5> Configures this middleware to forward the `Authorization` header from heimdall's response to the upstream service

=== Kubernetes Deployment

If you are using Traefik as https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/[Ingress Controller] or as https://gateway-api.sigs.k8s.io/[Gateway API] implementation in your kubernetes cluster, the required configuration is slightly different. The configuration of the entry point(s) stays the same, but the middleware needs to be deployed as a custom resource.

Here an example for a https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-middleware[`Middleware`] custom resource:

[source, yaml]
----
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata: # <1>
name: heimdall
namespace: heimdall
spec:
forwardAuth: # <2>
address: "http://heimdall.heimdall.svc.cluster.local:4456" # <3>
authResponseHeaders: # <4>
- Authorization
----
<1> The name and the namespace of the middleware. Both are set to `heimdall` here
<2> The type of the middleware, which is of type https://doc.traefik.io/traefik/middlewares/http/forwardauth/[`forwardAuth`]
<3> Configures this middleware to forward requests to the heimdall service. Here, the corresponding `Service` is named `heimdall` and is also located in the namespace named `heimdall`.
<4> Configures this middleware to forward the `Authorization` header from heimdall's response to the upstream service

How to add this middleware to the default middleware list of a particular endpoint depends on the method used to install Traefik. If helm is used, you can configure that list by making use of the following `values.yaml` file:

[source, yaml]
----
providers:
kubernetesCRD:
enabled: true # <1>
ports:
web: # <2>
middlewares:
- heimdall-heimdall@kubernetescrd # <3>
websecure: # <4>
middlewares:
- heimdall-heimdall@kubernetescrd
----
<1> To let traefik load `Middleware` resources, like defined above, traefik's `kubernetesCRD` provider must be enabled. Typically, it is enabled by default.
<2> Traefik's helm chart defines two entry points `web` for HTTP traffic and `websecure` for HTTPS traffic. Here we configure the `web` endpoint to use our middleware
<3> Reference to the `Middleware` resource, defined above. The general structure is `<middleware name>-<middleware namespace>@<provider>`. Since our middleware resource is loaded by the `kubernetescrd` provider, resides in the `heimdall` namespace, and is named `heimdall`, the reference `heimdall-heimdall@kubernetescrd` is used.
<4> Here we configure the `websecure` endpoint, which, as written above, is configured via helm chart for HTTPS traffic. The actual configuration is identical to the configuration for the `web` endpoint.

== Route-based Configuration with Docker

The integration option, described here makes use of the https://doc.traefik.io/traefik/providers/docker/[Docker Provider] for configuration discovery.
Expand Down Expand Up @@ -103,7 +150,40 @@ services:

If you have Traefik as Ingress Controller in your Kubernetes cluster, you can simply integrate heimdall globally as descibed in link:{{< relref "#_global_configuration" >}}[Global Configuration] chapter above and make use of the standard https://kubernetes.io/docs/concepts/services-networking/ingress/[Ingress resource].

There is also an option to have a route based configuration. In that case, you'll have to use Traefik proprietary https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-middleware[`Middleware`] and https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroute[`IngressRoute`] custom resources to define and use the https://doc.traefik.io/traefik/middlewares/http/forwardauth/[ForwardAuth] middleware.
If you are using traefik's proprietary https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-ingressroute[`IngressRoute`] custom resource instead of kubernetes standard https://kubernetes.io/docs/concepts/services-networking/ingress/[`Ingress`] one, you can also reference the https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/#kind-middleware[`Middleware`] resource locally. This option is shown in the snippet below.

[source, yaml]
----
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata: # <1>
name: demo-app
namespace: demo
spec:
entryPoints:
- web # <2>
routes:
- kind: Rule
match: Host(`demo-app.local`) && PathPrefix(`/`)
middlewares: # <3>
- name: heimdall
namespace: heimdall
services: # <4>
- kind: Service
name: demo-app
namespace: demo
port: app-port
----
<1> `metadata`, like name and the namespace of the `IngressRoute` resource
<2> The traefik entry points to attach this resource to. Here only `web` entry point is referenced
<3> List of the middlewares to be applied. Here the `Middleware` named `heimdall` in the namespace `heimdall` is referenced.
+
NOTE: By default, `IngressRoute` resources are not allowed to reference resources in namespaces different from the own namespace. If your `Middleware` resource, like also shown here, is deployed in another namespace, you have to allow that. If traefik is installed via helm, it can be achieved by setting `providers.kubernetesCRD.allowCrossNamespace` to `true` (See also https://doc.traefik.io/traefik/providers/kubernetes-crd/#allowcrossnamespace[here]).
<4> The reference to the `Service`, the requests should be forwarded to.

== Traefik as Gateway API implementation

If you have Traefik as https://gateway-api.sigs.k8s.io/[Gateway API] implementation in your Kubernetes cluster, you can simply integrate heimdall globally as descibed in link:{{< relref "#_kubernetes_deployment" >}}[Global Configuration] chapter above and make use of the standard https://gateway-api.sigs.k8s.io/api-types/httproute[`HTTPRoute`] resource.

== Additional Resources

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: '3.7'

services:
proxy:
image: traefik:2.11.0
image: traefik:3.0.0
ports:
- "9090:9090"
command: >
Expand Down
46 changes: 35 additions & 11 deletions examples/kubernetes/Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@ prometheus_version := '44.2.1'
loki_version := '2.8.9'
tempo_version := '0.16.8'
phlare_version := '0.1.2'
nginx_version := '9.7.7'
contour_version := '17.0.0'
emissary_version := '8.7.2'
haproxy_version := '0.14.4'
nginx_version := '4.10.1'
contour_version := '17.0.12'
emissary_version := '8.9.1'
haproxy_version := '0.14.6'
envoy_gw_version := 'v1.0.1'
metallb_version := '0.13.10'
traefik_version := '28.0.0'
metallb_version := '0.14.5'
certmanager_version := '1.14.5'
trustmanager_version := '0.9.2'

Expand All @@ -17,13 +18,15 @@ default_router := "contour"

setup-charts:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo add metallb https://metallb.github.io/metallb
helm repo add jetstack https://charts.jetstack.io
helm repo add dadrus https://dadrus.github.io/heimdall/charts
helm repo add datawire https://app.getambassador.io
helm repo add haproxy https://haproxy-ingress.github.io/charts
helm repo add traefik https://traefik.github.io/charts
helm repo update

## Installs Grafana
Expand Down Expand Up @@ -91,10 +94,10 @@ install-heimdall-pod-monitor:
install-observability-stack: install-grafana install-prometheus install-loki install-tempo install-phlare install-dashboards install-heimdall-pod-monitor

install-nginx-ingress-controller:
helm upgrade --install nginx-ingress-controller bitnami/nginx-ingress-controller \
-n nginx-ingress-controller --create-namespace \
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
-n nginx-controller --create-namespace \
--version {{nginx_version}} \
--set metrics.enabled=true \
--set controller.allowSnippetAnnotations=true \
--wait

install-contour-ingress-controller:
Expand All @@ -109,7 +112,7 @@ install-emissary-ingress-controller:
kubectl apply -f https://app.getambassador.io/yaml/emissary/${app_version}/emissary-crds.yaml
kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system
helm upgrade --install emissary datawire/emissary \
helm upgrade --install emissary datawire/emissary-ingress \
-n emissary-controller --create-namespace \
--version {{emissary_version}}

Expand All @@ -119,7 +122,7 @@ install-emissary-ingress-controller:


install-haproxy-ingress-controller:
helm upgrade --install haproxy-controller haproxy/haproxy \
helm upgrade --install haproxy-controller haproxy/haproxy-ingress \
-n haproxy-controller --create-namespace \
--version {{haproxy_version}} \
-f haproxy/helm-values.yaml \
Expand All @@ -133,6 +136,19 @@ install-envoy-gateway:

kubectl apply -f envoygw/gateway.yaml

install-traefik global_mw="true":
#!/usr/bin/env bash
valuesFile=$({{global_mw}} && echo global-mw-helm-values.yaml || echo helm-values.yaml)
kubectl apply -f traefik/certificate.yaml

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml

helm upgrade --install traefik traefik/traefik \
-n traefik \
--version {{traefik_version}} \
-f traefik/${valuesFile} \
--wait

install-lb:
#!/usr/bin/env bash
Expand Down Expand Up @@ -183,7 +199,7 @@ install-heimdall router=default_router:
extraArgs='extraArgs={--envoy-grpc}'
fi

helm upgrade --install heimdall dadrus/heimdall \
helm upgrade --install heimdall ../../charts/heimdall \
-n heimdall \
-f quickstarts/heimdall/config.yaml \
-f quickstarts/heimdall/helm-values.yaml \
Expand All @@ -197,6 +213,8 @@ install-heimdall router=default_router:
elif [ "{{router}}" == "envoygw" ]; then
kubectl apply -f quickstarts/heimdall/backend-tls-policy.yaml
kubectl apply -f quickstarts/heimdall/envoygw-security-policy.yaml
elif [ "{{router}}" == "traefik" ]; then
kubectl apply -f quickstarts/heimdall/heimdall-middleware.yaml
fi

create-cluster:
Expand All @@ -214,6 +232,12 @@ install-emissary-decision-demo: setup-cluster install-emissary-ingress-controlle

install-envoygw-decision-demo: setup-cluster install-envoy-gateway (install-heimdall "envoygw") (install-echo-service "envoygw")

install-traefik-ingress-decision-demo: setup-cluster install-traefik (install-heimdall "traefik") (install-echo-service "traefik-ingress")

install-traefik-ingress-route-decision-demo: setup-cluster (install-traefik "false") (install-heimdall "traefik") (install-echo-service "traefik-ingress-route")

install-traefik-gw-decision-demo: setup-cluster install-traefik (install-heimdall "traefik") (install-echo-service "traefik-gw")

delete-cluster:
kind delete clusters {{cluster_name}}

2 changes: 1 addition & 1 deletion examples/kubernetes/emissary/listener.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: getambassador.io/v3alpha1
kind: Listener
metadata:
name: emissary-tls-ingress-listener
namespace: emissary-ingress-controller
namespace: emissary-controller
spec:
port: 8443
protocol: HTTPS
Expand Down
8 changes: 4 additions & 4 deletions examples/kubernetes/kind/kind.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
image: kindest/node:v1.28.0@sha256:9f3ff58f19dcf1a0611d11e8ac989fdb30a28f40f236f59f0bea31fb956ccf5c
image: kindest/node:v1.29.2@sha256:acc9e82a5a5bd3dfccfd03117e9ef5f96b46108b55cd647fb5e7d0d1a35c9c6f
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
- role: worker
image: kindest/node:v1.28.0@sha256:9f3ff58f19dcf1a0611d11e8ac989fdb30a28f40f236f59f0bea31fb956ccf5c
image: kindest/node:v1.29.2@sha256:acc9e82a5a5bd3dfccfd03117e9ef5f96b46108b55cd647fb5e7d0d1a35c9c6f
- role: worker
image: kindest/node:v1.28.0@sha256:9f3ff58f19dcf1a0611d11e8ac989fdb30a28f40f236f59f0bea31fb956ccf5c
image: kindest/node:v1.29.2@sha256:acc9e82a5a5bd3dfccfd03117e9ef5f96b46108b55cd647fb5e7d0d1a35c9c6f
- role: worker
image: kindest/node:v1.28.0@sha256:9f3ff58f19dcf1a0611d11e8ac989fdb30a28f40f236f59f0bea31fb956ccf5c
image: kindest/node:v1.29.2@sha256:acc9e82a5a5bd3dfccfd03117e9ef5f96b46108b55cd647fb5e7d0d1a35c9c6f
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ metadata:
haproxy-ingress.github.io/auth-url: "https://heimdall.heimdall.svc.cluster.local:4456"
haproxy-ingress.github.io/auth-headers-succeed: "authorization"
haproxy-ingress.github.io/headers: |
X-Forwarded-Uri: %[baseq]
X-Forwarded-Uri: %[pathq]
X-Forwarded-Method: %[method]
X-Forwarded-Host: %[req.hdr(host)]
spec:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: echo-app
namespace: quickstarts
spec:
parentRefs:
- name: traefik-gateway
namespace: traefik
hostnames:
- "echo-app.local"
rules:
- backendRefs:
- group: ""
kind: Service
name: echo-app
port: 8080
weight: 1
matches:
- path:
type: PathPrefix
value: /
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- http_route.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: echo-app
namespace: quickstarts
labels:
app.kubernetes.io/name: echo-app
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`echo-app.local`) && PathPrefix(`/`)
middlewares:
- name: heimdall
namespace: heimdall
services:
- kind: Service
name: echo-app
namespace: quickstarts
port: app-port
tls:
secretName: echo-app
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
- ingress-route.yaml
Loading

0 comments on commit 303095e

Please sign in to comment.