I use cookies in order to optimize my website and continually improve it. By continuing to use this site, you are agreeing to the use of cookies.
You can find an Opt-Out option and more details on the Privacy Page!

Install kubernetes ingress

In this blogpost I’ll show how to setup a nginx-ingress controller and how to deploy the first application that uses the ingress. Additionally I’ll show how to setup a DigitalOcean LoadBalancer that handles requests on port 80 and 443 and redirects these requests to the ingress controller.

Architecture of Ingress

Ingress architecture

As you should see in the picture there is a Kubernetes Service deployed in the kubernetes cluster that receives requests from outside the network. In our example we use the ports 30080 and 30443. This service balances the received traffic to the ingress pods. These pods are nginx servers that are configured to forward traffic by a hostname and path variable to other services. This services are the load balancers for our applications deployed in the cluster that should be exposed to the outside. They will receive the traffic from the ingress pods and forward them to the application pods.

How does the Ingress pods know how to proxy traffic to the correct services. Therefore in kubernetes a resource called ingress is defined. The entries specified in ingress define how traffic should be proxied. An example ingress looks like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  namespace: application
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: application.koudingspawn.de
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx-service
          servicePort: 80

This ingress specifies, that traffic received with host header application.koudingspawn.de should be forwarded to the nginx-service on port 80. After creating this resource the nginx-ingress controller will check the kubernetes-api and receive a new ingress configuration. With this information he reconfigures the nginx-ingress controller to balance incomming traffic with a host header application.koudingspawn.de to the service called nginx-service. It is important to know, that the ingress resource must be specified in the same namespace as the service is deployed that should receive the incomming traffic.

How to setup nginx-ingress

Setup of nginx-ingress is very easy, we only need to run some kubectl apply commands that will configure the nginx-ingress controller. This part is a copy of the https://github.com/kubernetes/ingress-nginx/tree/master/deploy page that describes how to deploy nginx-ingress. The cluster created in the previous posts is a cluster without RBAC access, so we must use the without-rbac.yml file to create the deployment:

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \
    | kubectl apply -f -

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \
    | kubectl apply -f -

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \
    | kubectl apply -f -

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \
    | kubectl apply -f -

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \
    | kubectl apply -f -

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml \
    | kubectl apply -f -

This should start in the namespace ingress-nginx two pods:

kubectl get pods -n ingress-nginx
NAME                                       READY     STATUS    RESTARTS   AGE
default-http-backend-7c7764588c-5zn2f      1/1       Running   0          1m
nginx-ingress-controller-886445d9c-wxsp9   1/1       Running   0          1m

The first pod is the default backend, to which requests are forwarded that cannot be assigned. This backend has to comply to two prerequisites:

  1. It must return 200 on /healthz
  2. It must return 404 on /

You could replace the default backend by your own implementation this is very easy. The second pod that is deployed is the nginx-ingress-controller that receives the requests and proxies them to the services defined in the ingress configuration resources.

Now we need to deploy the service that exposes the nginx-ingress-controller on ports 30080 and 30443 as described already in the tutorial:

kind: Service
apiVersion: v1
metadata:
  name: ingress-lb
  namespace: ingress-nginx
spec:
  selector:
    app: ingress-nginx
  type: NodePort
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30080
  - name: https
    protocol: TCP
    port: 443
    targetPort:  443
    nodePort: 30443

Thats all, now we can try to access it by running curl against this resources:

curl http://{ ip }:30080

This should produce the following output: default backend - 404

Setup your first application that uses ingress

This application is a simple nginx server that receives requests on port 80 and returns the default nginx page. Therefore we create a namespace called application, a service called nginx-service and a deployment called nginx-deployment:

apiVersion: v1
kind: Namespace
metadata:
  name: application
---

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  namespace: application
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  namespace: application
spec:
  selector:
    app: nginx
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

Now you should see the following output after you applied the configuration.

$ kubectl get pods -n application
NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-569477d6d8-8q7kw   1/1       Running   0          1m
nginx-deployment-569477d6d8-hms7z   1/1       Running   0          1m
nginx-deployment-569477d6d8-ldf5s   1/1       Running   0          1m

$ kubectl get svc -n application -o wide
NAME            TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE       SELECTOR
nginx-service   ClusterIP   10.32.0.103   <none>        80/TCP    5m        app=nginx

The next step is to route traffic to the nginx-service by adding the following ingress configuration:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  namespace: application
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: application.koudingspawn.de
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx-service
          servicePort: 80

As already specified traffic that arrives with host header application.koudingspawn.de should be proxied from the nginx-ingress-controller to the nginx-service and from there to the nginx pods.

Now you should see the following output when running curl with host header application.koudingspawn.de:

$ curl http://{public_ip}:30080/ -H "Host: application.koudingspawn.de"

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

How to setup a LoadBalancer on DigitalOcean

This step is only an appendix, I tried to run the steps but my LoadBalancer doesn’t proxy traffic to the nodes. He was healthy and the droplets were also healthy but it won’t forward traffic.

We will specify in our LoadBalancer that traffic from port 80 should be forwarded to port 30080 on each of the nodes and traffic on port 443 should be forwarded to port 30443. Ports 30080 and 30443 are the ports the ingress-lb service watches for traffic on. Additionally we specify a health check on port 30080, there we use the features of the default backend to return status code 200 on /healthz. Here is the terraform script we use to generate the LoadBalancer:

resource "digitalocean_loadbalancer" "public" {
  name = "worker-loadbalancer"
  region = "fra1"

  forwarding_rule {
    entry_port = 80
    entry_protocol = "tcp"

    target_port = 30080
    target_protocol = "tcp"
  }

  forwarding_rule {
    entry_port = 443
    entry_protocol = "tcp"

    target_port = 30443
    target_protocol = "tcp"
  }

  healthcheck {
    port = 30080
    protocol = "http"
    path = "/healthz"
  }

  droplet_tag = "kube-worker"
}

Our droplets are tagged with the kube-worker tag, so new upcomming droplets will join the LoadBalancer when their healthcheck on Port 30080/healthz returns a status of 200. This will happen when the kubelet and kube-proxy running on the nodes are comming up.

The last step is to connect a domain to the LoadBalancer in our case we do this with CloudFlare:

resource "cloudflare_record" "worker_lb" {
  domain = "${var.cloudflare_domain}"
  name = "application"
  type = "A"
  value = "${digitalocean_loadbalancer.public.ip}"
}
Björn Wenzel

Björn Wenzel

My name is Björn Wenzel. I’m a DevOps with interests in Kubernetes, CI/CD, Spring and NodeJS.