🎉 Announcing new lower pricing — up to 40% lower costs for Cloud Servers and Cloud SQL! Read more →

Auto-signing Kubernetes Server Certificates

Previously, we explained how to securely install the Kubernetes Metrics Server. However, the Kubernetes Controller Manager will only approve certificate signing requests for client certificates necessary to connect to the API server. Signing requests for server certificates requires manual approval, both initially and for each renewal.

Which tends to be rather inconvenient.

This default behaviour is meant to protect against a compromised server generating arbitrary certificates and mounting Man in the Middle attacks within the cluster. But the reasoning for this default behaviour is a bit outdated, so we can enable auto-signing now, as long as we’re careful.

Signing Kubelet’s certificate requests automatically

There are third party operators available which will sign the server requests for you. The one we are using is Kubelet Rubber Stamp. This approves a kubelet certificate signing request (CSR) as long as the requestor id passes a Subject Access Review which ensures the node has the correct authorization to request certificates.

Rubber Stamp will only accept certificate requests where the Common Name matches the authorized requestor id, the Common Name has a system:node: prefix and the Organisation is system:nodes. The requestor id (in our case, the hostname) comes from kubelet’s client certificate, which is already authorised and signed so we can trust it. And those common name prefixes mean that even if an attacker can somehow obtain a server certificate with an arbitrary name, they are clearly only kubelet certificates.

This all prevents a compromised server generating certificates for arbitrary services within the cluster, so the man is kept out of the middle.

Let’s enable the auto-signer.


Clone the Kubelet Rubber Stamp repository and apply the deploy directory

$ git clone git@github.com:kontena/kubelet-rubber-stamp.git /tmp/stamp
$ kubectl apply -f /tmp/stamp/deploy

Once installed, the operator will approve all outstanding server CSRs and any new ones that kubelet creates during certificate renewal.

You can view the logs to check the certificates are being signed as expected

$ kubectl -n kube-system logs deployment/kubelet-rubber-stamp
I0916 14:41:02.068037       1 main.go:19] Go Version: go1.11.13
I0916 14:41:02.068320       1 main.go:20] Go OS/Arch: linux/amd64
I0916 14:41:02.068503       1 main.go:21] operator-sdk Version: v0.4.0
I0916 14:41:02.478254       1 main.go:49] Registering Components.
I0916 14:41:02.483060       1 main.go:61] Starting the Cmd.
I0916 14:41:02.784772       1 certificatesigningrequest_controller.go:84]
    Reconciling CertificateSigningRequest /csr-hnb48
I0916 14:41:02.792646       1 certificatesigningrequest_controller.go:131]
    approving csr csr-hnb48 with SANs: [srv-krwu8], IP Addresses:[]
I0916 14:41:02.805376       1 certificatesigningrequest_controller.go:84]
    Reconciling CertificateSigningRequest /csr-554ml
I0916 14:41:02.810093       1 certificatesigningrequest_controller.go:131]
    approving csr csr-554ml with SANs: [srv-krwu8], IP Addresses:[]
I0916 14:41:02.819700       1 certificatesigningrequest_controller.go:84]
    Reconciling CertificateSigningRequest /csr-6qm2f
I0916 14:41:02.819892       1 certificatesigningrequest_controller.go:101]
    CSR already has a certificate, ignoring

Give Brightbox a go for free

If you want to play with Kubernetes, you can sign up for Brightbox Cloud in just a couple of minutes and use your £50 free credit to give it a go.

Managed Kubernetes services

If instead you want us to run Kubernetes for you, or anything else for that matter, we offer hands-on support and managed services too. Drop us a line.

Get started with Brightbox Sign up takes just two minutes...