Contents

Deploy an app behind a Load Balancer on Kubernetes

This guide takes you through deploying an example application on Kubernetes, using a Brightbox Load Balancer with a Let’s Encrypt certificate.

Requirements

You’ll need to have deployed a Kubernetes cluster along with our Kubernetes Cloud Controller plugin. The easiest way to do this is with our Terraform configuration.

Connect to your Kubernetes cluster

If you’re using our Terraform configuration, the master output is the public IP address of the Kubernetes master server. You can SSH into this server using your SSH key:

$ terraform output master
cip-dwj6c.gb1.brightbox.com

$ ssh ubuntu@cip-dwj6c.gb1.brightbox.com

Last login: Tue Aug 14 15:56:46 2018 from ...
ubuntu@srv-5cbsi:~$

And use kubectl on the master to inspect the cluster:

ubuntu@srv-5cbsi:~$ kubectl get nodes
NAME        STATUS    ROLES     AGE       VERSION
srv-5cbsi   Ready     master    21d       v1.11.2
srv-q5t2m   Ready     <none>    21d       v1.11.2
srv-u8dbu   Ready     <none>    21d       v1.11.2

See here we have one master server and two other nodes.

Deploy the application

First we’ll create a deployment, which will handle creating and managing a pod for our application. We’ll use an example “hello world” rails app image that runs a rails http server and listens on port 3000.

Let’s define the deployment in a file called hello-world-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
        - name: app
          image: brightbox/rails-hello-world
          ports:
            - name: web
              containerPort: 3000
              protocol: TCP

and create it with kubectl:

$ kubectl create -f hello-world-deployment.yaml 
deployment.apps/hello-world created

And you can confirm the deployment exists and is deploying the app:

$ kubectl describe deployments

Name:                   hello-world
Namespace:              default
CreationTimestamp:      Fri, 21 Sep 2018 11:31:18 +0100
Labels:                 <none>
Annotations:            deployment.kubernetes.io/revision=1
Selector:               app=hello-world
Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=hello-world
  Containers:
   app:
    Image:        brightbox/rails-hello-world
    Port:         3000/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      False   MinimumReplicasUnavailable
  Progressing    True    ReplicaSetUpdated
OldReplicaSets:  <none>
NewReplicaSet:   hello-world-7df8d96f69 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  11s   deployment-controller  Scaled up replica set hello-world-7df8d96f69 to 1

The deployment has created a pod:

$ kubectl get pods
NAME                           READY     STATUS              RESTARTS   AGE
hello-world-7df8d96f69-5xp8w   0/1       ContainerCreating   0          30s

and soon after you’ll find the pod has successfully pulled in the container image and is running:

$ kubectl describe pod/hello-world-7df8d96f69-5xp8w
Name:               hello-world-7df8d96f69-5xp8w
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               srv-q5t2m/10.243.20.10
Start Time:         Fri, 21 Sep 2018 11:31:18 +0100
Labels:             app=hello-world
                    pod-template-hash=3894852925
Annotations:        cni.projectcalico.org/podIP=192.168.2.2/32
Status:             Running
IP:                 192.168.2.2
Controlled By:      ReplicaSet/hello-world-7df8d96f69
Containers:
  app:
    Container ID:   docker://df78eb2abe52ef1768c57e57ff8b445d6fe65c4b2d6dda2eb1fbf81137658f7f
    Image:          brightbox/rails-hello-world
    Image ID:       docker-pullable://brightbox/rails-hello-world@sha256:4d22266b00bd649b28bf06cec85849c87a0902552777b2ca38cd3734607bee04
    Port:           3000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 21 Sep 2018 11:31:57 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-z4h9x (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-z4h9x:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-z4h9x
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From                Message
  ----    ------     ----  ----                -------
  Normal  Scheduled  7m    default-scheduler   Successfully assigned default/hello-world-7df8d96f69-5xp8w to srv-q5t2m
  Normal  Pulling    7m    kubelet, srv-q5t2m  pulling image "brightbox/rails-hello-world"
  Normal  Pulled     6m    kubelet, srv-q5t2m  Successfully pulled image "brightbox/rails-hello-world"
  Normal  Created    6m    kubelet, srv-q5t2m  Created container
  Normal  Started    6m    kubelet, srv-q5t2m  Started container

So now the application is running and is reachable from within the cluster. You can get the pod ip address and connect to port 3000 from the master and you’ll get a response:

ubuntu@srv-5cbsi:~$ curl -I http://192.168.2.2:3000
HTTP/1.1 200 OK
Set-Cookie: _hello_world_session=...
X-Request-Id: e108ce2d-bd7d-43d5-943a-5037e203157c
X-Runtime: 0.002707

But it’s not reachable from outside the cluster. That’s where a Load Balancer comes in.

Expose the app using a Load Balancer

Let’s define the load balancer service with a file called hello-world-service.yaml:

kind: Service
apiVersion: v1
metadata:
  name: hello-world
  annotations:
    service.beta.kubernetes.io/brightbox-load-balancer-healthcheck-request: /
spec:
  type: LoadBalancer
  selector:
    app: hello-world
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: web

and create the service with kubectl:

$ kubectl create -f hello-world-service.yaml 
service/hello-world created

Within a minute the load balancer should be confirmed up and running:

$ kubectl get service/hello-world
NAME          TYPE           CLUSTER-IP       EXTERNAL-IP                                         PORT(S)        AGE
hello-world   LoadBalancer   172.30.158.247   109.107.37.214,2a02:1348:ffff:ffff::6d6b:25d6,...   80:30463/TCP   9m

We can see Kubernetes has created the load balancer and mapped a new Cloud IP address to it.

Now it’s reachable from anywhere:

$ curl -I http://109.107.37.214
HTTP/1.1 200 OK
Set-Cookie: _hello_world_session...
X-Request-Id: 7df7456e-1a3e-4599-b541-28e918f1fb36
X-Runtime: 0.002784

And via IPv6 of course, as our Cloud IP addresses are both IPv4 and IPv6 now:

$ curl -I http://[2a02:1348:ffff:ffff::6d6b:25d6]
HTTP/1.1 200 OK
Set-Cookie: _hello_world_session...
X-Request-Id: 8cbc90ab-65f4-4203-8a2d-8ae48bd1f512
X-Runtime: 0.002928

And you can see the load balancer in Brightbox Manager, named so you can recognise it as part of the Kubernetes cluster.

Enabling SSL with a Let’s Encrypt certificate

Now let’s enable SSL acceleration on the Load Balancer and have it get a Let’s Encrypt certificate for us. It’s really easy!

Firstly you’ll need to point a domain name at the cloud IP. In this example, the allocated Cloud IP is 109.107.37.214. So I’ve setup helloworld.fdns.uk to resolve to that IP.

Then we need to modify the hello-world service and add a couple of extra options. One new annotations to specify the domain name and and an additional listener on port 443. If you want to specify multiple domains for the certificate, just comma separate them here.

Notice that we’ve also specified the exact Cloud IP we want now, with loadBalancerIP. Now that we have a domain pointing at this IP we want to make sure we use the same IP for this service in future, even if we recreate it from scratch. Otherwise a new random one will be created and the IP will change.

kind: Service
apiVersion: v1
metadata:
  name: hello-world
  annotations:
    service.beta.kubernetes.io/brightbox-load-balancer-healthcheck-request: /
    service.beta.kubernetes.io/brightbox-load-balancer-ssl-domains: helloworld.fdns.uk

spec:
  type: LoadBalancer
  loadBalancerIP: 109.107.37.214
  selector:
    app: hello-world
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: web
    - name: https
      protocol: TCP
      port: 443
      targetPort: web

Then just apply it:

$ kubect apply -f hello-world-service.yaml 
service/hello-world configured

And check on it to see the new port has been added:

$ kubectl get service/hello-world
NAME          TYPE           CLUSTER-IP       EXTERNAL-IP          PORT(S)                      AGE
hello-world   LoadBalancer   172.30.158.247   109.107.37.214,...   80:30463/TCP,443:30507/TCP   2h

And you’ll be able to reach the service via HTTPS now:

$ curl -I https://helloworld.fdns.uk
HTTP/1.1 200 OK
Set-Cookie: _hello_world_session...
X-Request-Id: 0859268d-168d-4ca2-9297-2bf4dc2118a0
X-Runtime: 0.003849

And you don’t even have to think about expiring certificates as the Brightbox Load Balancer service will handle renewing the certificate it automatically every 90 days.

Scale it up

Finally, let’s pretend our hello world application suddenly became popular and we want to add more resources for it. This Kubernetes cluster already has 2 nodes, so we can scale up the deployment to 2 and double the resources available to it.

So we take the deployment, currently configured to run just 1 pod:

$ kubectl get deployment
NAME          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
hello-world   1         1         1            1           4h

And we tell it to scale up to 2:

$ kubectl scale --replicas=2 deployment/hello-world
deployment.extensions/hello-world scaled

$ kubectl get deployment
NAME          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
hello-world   2         2         2            1           4h

A second pod starts building:

$ kubectl get pods
NAME                           READY     STATUS              RESTARTS   AGE
hello-world-7df8d96f69-2wlzr   0/1       ContainerCreating   0          17s
hello-world-7df8d96f69-5xp8w   1/1       Running             0          4h

And once it’s ready, we’re done! The second pod is automatically added to the load balancer service ready to receive traffic.

Last updated: 29 Oct 2018 at 15:38 UTC

Try Brightbox risk-free with £20 free credit Sign up takes just two minutes...