How To Renew Kubernetes Certificates

Written by: Bagus Facsi Aginsa
Published at: 25 Nov 2023


If you manage a Kubernetes cluster, you will need to renew the certificate once a year. Lucky for us Kubernetes provides an easy way to renew all the certificates needed.

The single command needed to renew Kubernetes certificates is kubeadm certs renew all. Let’s see the details of how it is done in this tutorial.

Sudo Privileges

Before starting, we make sure that we will have no permission issues on the installation and configuration. Go to your kubernetes master node and type this command

sudo su

Backup Old Certificates

Before we renew the certificate, it is better to back up our existing configuration & certificate so we can use it later if the renewing process fails. We can copy them to /tmp folder on your kubernetes master node.

mkdir /tmp/k8s-backup
cp /etc/kubernetes/*.conf /tmp/k8s-backup
cp -r /etc/kubernetes/pki /tmp/k8s-backup

Renew Certificates

First, check the expiration date of our cluster certificates by typing this command on your kubernetes master node:

kubeadm certs check-expiration

You will get the expiration date of each component like this one:

root@k8s-master:/home/bagus# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Nov 21, 2024 22:53 UTC   364d            ca                      no
apiserver                  Nov 21, 2024 22:53 UTC   364d            ca                      no
apiserver-etcd-client      Nov 21, 2024 22:53 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Nov 21, 2024 22:53 UTC   364d            ca                      no
controller-manager.conf    Nov 21, 2024 22:54 UTC   364d            ca                      no
etcd-healthcheck-client    Nov 21, 2024 22:53 UTC   364d            etcd-ca                 no
etcd-peer                  Nov 21, 2024 22:53 UTC   364d            etcd-ca                 no
etcd-server                Nov 21, 2024 22:53 UTC   364d            etcd-ca                 no
front-proxy-client         Nov 21, 2024 22:53 UTC   364d            front-proxy-ca          no
scheduler.conf             Nov 21, 2024 22:54 UTC   364d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Nov 19, 2033 22:53 UTC   9y              no
etcd-ca                 Nov 19, 2033 22:53 UTC   9y              no
front-proxy-ca          Nov 19, 2033 22:53 UTC   9y              no

To renew our certificates, we just need this one command:

kubeadm certs renew all

Renew Kubernetes Certificate Command

You can confirm that the certificate is renewed by typing

kubeadm certs check-expiration

You will see the expiration date is updated

root@k8s-master:/home/bagus# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Nov 21, 2024 23:02 UTC   364d            ca                      no
apiserver                  Nov 21, 2024 23:02 UTC   364d            ca                      no
apiserver-etcd-client      Nov 21, 2024 23:02 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Nov 21, 2024 23:02 UTC   364d            ca                      no
controller-manager.conf    Nov 21, 2024 23:02 UTC   364d            ca                      no
etcd-healthcheck-client    Nov 21, 2024 23:02 UTC   364d            etcd-ca                 no
etcd-peer                  Nov 21, 2024 23:02 UTC   364d            etcd-ca                 no
etcd-server                Nov 21, 2024 23:02 UTC   364d            etcd-ca                 no
front-proxy-client         Nov 21, 2024 23:02 UTC   364d            front-proxy-ca          no
scheduler.conf             Nov 21, 2024 23:02 UTC   364d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Nov 19, 2033 22:53 UTC   9y              no
etcd-ca                 Nov 19, 2033 22:53 UTC   9y              no
front-proxy-ca          Nov 19, 2033 22:53 UTC   9y              no

To make sure our control plane is using the newest certificate, we need to restart the kube-apiserver, kube-controller-manager, kube-scheduler, and etcd, so they read the new certificate file. To do that, we can move their manifest to temporary locations one by one.

Restart API Server Pod

Let’s start with kube-apiserver. Because we are messing with the control plane, the kubectl command will not work properly here. So we can check their status using the containerd command like this

crictl -r unix:///run/containerd/containerd.sock ps

First, make sure the API server is running

root@k8s-master:/home/bagus# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-apiserver
23e7455c1893e       7fe0e6f37db33       14 minutes ago      Running             kube-apiserver              0                   f1c0570fe0d86       kube-apiserver-k8s-master

Move the manifest to a temporary folder

mv /etc/kubernetes/manifests/kube-apiserver.yaml /tmp

After a while, you will notice the API server is missing

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-apiserver
root@k8s-master:/etc/kubernetes/manifests#

Move the manifest back from the temporary folder to the original place

mv /tmp/kube-apiserver.yaml /etc/kubernetes/manifests

After a while, the API server will back to running status

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-apiserver
b3d09c446e612       7fe0e6f37db33       9 seconds ago       Running             kube-apiserver              0                   f06a71451213b       kube-apiserver-k8s-master

Restart Controller Manager Pod

First, make sure the controller manager is running

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-controller-manager
1916adb941070       d058aa5ab969c       3 minutes ago        Running             kube-controller-manager     1                   907fda45461ad       kube-controller-manager-k8s-master

Move the manifest to the temporary folder

mv /etc/kubernetes/manifests/kube-controller-manager.yaml /tmp

After a while, you will notice the controller manager is missing

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-controller-manager
root@k8s-master:/etc/kubernetes/manifests#

Move the manifest back from the temporary folder to the original place

mv /tmp/kube-controller-manager.yaml /etc/kubernetes/manifests

After a while, the controller manager will back to running status

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-controller-manager
bc83a56a4b3d8       d058aa5ab969c       10 seconds ago       Running             kube-controller-manager     0                   61c551ffacaed       kube-controller-manager-k8s-master

Restart Scheduler Pod

First, make sure the Scheduler is running

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-scheduler
023b5f2a3ab60       e3db313c6dbc0       5 minutes ago        Running             kube-scheduler              1                   391207efd986d       kube-scheduler-k8s-master

Move the manifest to the temporary folder

mv /etc/kubernetes/manifests/kube-scheduler.yaml /tmp

After a while, you will notice the scheduler is missing

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-scheduler
root@k8s-master:/etc/kubernetes/manifests#

Move the manifest back from the temporary folder to the original place

mv /tmp/kube-scheduler.yaml /etc/kubernetes/manifests

After a while, the scheduler will back to running status

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep kube-scheduler
499482089ba6a       e3db313c6dbc0       8 seconds ago        Running             kube-scheduler              0                   db2c517b6d8a7       kube-scheduler-k8s-master

Restart etcd Pod

First, make sure the etcd is running

root@k8s-master:/home/bagus# crictl -r unix:///run/containerd/containerd.sock ps | grep etcd
1ec1f23d6afd9       73deb9a3f7025       2 days ago          Running             etcd                        0                   33d04c480952c       etcd-k8s-master

Move the manifest to the temporary folder

mv /etc/kubernetes/manifests/etcd.yaml /tmp

After a while, you will notice the etcd is missing

root@k8s-master:/home/bagus# crictl -r unix:///run/containerd/containerd.sock ps | grep etcd
root@k8s-master:/home/bagus#

Move the manifest back from the temporary folder to the original place

mv /tmp/etcd.yaml /etc/kubernetes/manifests

Notes!

I notice that etcd pod need more time to start compared to other controller pod, it takes up to ~5 minutes in my case.

After a while, the etcd will back to running status

root@k8s-master:/etc/kubernetes/manifests# crictl -r unix:///run/containerd/containerd.sock ps | grep etcd
91ecdfacd0ff2       73deb9a3f7025       12 seconds ago      Running             etcd                        0                   0b72aff317cb5       etcd-k8s-master

Done! Congratulations, now your Kubernetes running using the newest certificates.

If you have multiple master nodes on your Kubernetes Cluster, you can do the same steps that are explained in this article and execute them on each master node, starting from renewing the certificates until restarting the control plane pod.