If you are working with Kubernetes a lot, you have probably built several basic clusters for learning purposes using kubeadm and the documentation here. As you start exploring topics like RBAC Roles and Pod Security Policies, you will soon notice that only one user was created in each cluster. And since that user has the cluster-admin role, it can do anything in any namespace. To try out many of the Kubernetes security-related concepts in your clusters, you will need to add Kubernetes users that are not cluster administrators. For your basic clusters, you probably don’t have integration set up with an external system to add and authenticate users. And as the Kubernetes docs note here: Kubernetes does not have objects which represent normal user accounts. Normal users cannot be added to a cluster through an API call.
This blog post will show you how to create new Kubernetes users in your clusters, focusing on clusters created using kubeadm.
For clusters created using other deployment tools, you may need to modify the instructions. While there are several ways to authenticate users, I will use the X509 certificate approach in this blog.
My environment uses Ubuntu 16.04 LTS, but the instructions in this blog should generally work for any similar Linux environment. Since I will be using OpenSSL, if you want to follow along with this blog you will need to install OpenSSL on your system. You will also need root or sudo access for some commands.
I’ll be doing most of the work on the master node of my Kubernetes cluster, so I’ll SSH into the master node.
First, check if OpenSSL is installed:
If OpenSSL is installed you will get a response showing the version. Otherwise, you will need to install OpenSSL:
sudo apt-get update sudo apt-get install openssl
Alternatively, I could do my work on a client workstation or VM with kubectl and the kubeconfig file for my cluster installed and configured. In that case, I would also need to copy the Kubernetes ca.key and ca.crt files from /etc/kubernetes/pki/ on a master node so they are available locally when I sign the new certificate in one of the steps. If you copy the ca.key file, remember to take appropriate security precautions to prevent unauthorized access to that key file.
Add a new Kubernetes user account
I will create a new user named user-2, and I will put that user in a group named dev. To keep things organized, I’ll create a directory in my file system and do my work in that directory. The name of the directory is not important; I’ll create a directory named kube-users for this example:
mkdir kube-users cd kube-users
Create and sign a certificate for the new user
First, create a private key:
openssl genrsa -out user-2.key 2048
Next, create a certificate signing request. The value of CN is the user name, and the value of O is the group name:
openssl req -new -key user-2.key -out user-2.csr -subj "/CN=user-2/O=dev"
Finally, sign the CSR to create a certificate signed by the Kubernetes CA. The resulting certificate in this example will not expire for 720 days.
sudo openssl x509 -req -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -days 720 -in user-2.csr -out user-2.crt
Add the user to your kubeconfig
Now that I have a certificate created, I can add credentials and a context for the new user to my kubeconfig file. I can do this on the master node, or I can copy the user’s certificate to another workstation or VM that I am using as a kubectl client. Wherever I do this work, I should use an existing kubeconfig file that is up to date. That way if I later copy it to another machine or VM I will not overwrite or lose newer kubeconfig settings I have created there. For this example, I will do the work on the master node and then copy the resulting kubeconfig file to other clients as needed.
Create the new user’s credentials
I’ll use the approach of embedding the certificate data in the kubeconfig file so that it will still be available if I copy the file to another location. I must use an absolute path to the user’s crt and key files because a relative path such as ~/kube-users/user-2.crt will cause an error. If you are following along, you’ll need to change my-login to the name of your account.
kubectl config set-credentials user-2 --client-certificate='/home/my-login/kube-users/user-2.crt' --client-key='/home/my-login/kube-users/user-2.key' --embed-certs=true
Create a context for the new user
I will first need to get the name used for my cluster in my kubeconfig file.
kubectl config get-clusters
There may be more than one cluster referenced in my kubeconfig file, especially if I do this step on a client machine rather than a master node. If the command returns more than one cluster name, then I need to identify the name used for the target cluster. I will then use the name of the target cluster as the value of
--cluster argument in the
kubectl config set-context command. I also need to provide a name for the new context. In this example, my cluster name happens to be kubernetes, so I will name my new context user-2@kubernetes. Also note that the
--namespace argument is optional and if it not specified, the context will use the default namespace. In this example, I will specify the development namespace. That means that when I am using this context, any kubectl commands I execute will use the development namespace by default.
kubectl config set-context user-2@kubernetes --cluster=kubernetes --user=user-2 --namespace=development
Test the new Kubernetes user account
Set up for testing
I’ll use a context with the cluster-admin ClusterRole while I set things up for my testing.
Make sure the target namespace exists
If the namespace that I specified for the new user’s context does not exist, I’ll need to create it:
kubectl create ns development
Configure permissions in the target namespace
Unless I already set up permissions for the dev group that the new user belongs to, the new user will not yet have any permissions granted in the cluster. For this example, let’s assume that the dev group does not have any permissions granted in the development namespace.
I can create a RoleBinding in the development namespace for the new user or for a group the user belongs to. In this case, I’ll create a RoleBinding for the dev group, and user-2 will inherit permissions from that group and RoleBinding. I’ll use the ClusterRole edit to create a RoleBinding named dev-edit-rolebinding for the dev group:
kubectl -n development create rolebinding dev-edit-rolebinding --clusterrole=edit --group=dev
Try out the new account
First, I’ll change the context to switch to the new user account:
kubectl config use-context user-2@kubernetes
Next, I’ll try creating a pod in the development namespace:
kubectl -n development run test-nginx --restart Never --image nginx kubectl -n development get pods NAME READY STATUS RESTARTS AGE test-nginx 1/1 Running 0 25s
After that, I’ll try creating a pod in another namespace where the new user does not have any permissions granted. In the cluster I’m using for this example, I have another namespace named test1 where neither the dev group nor user-2 have any permissions granted:
kubectl -n test1 run test-nginx --restart Never --image nginx Error from server (Forbidden): pods is forbidden: User "user-2" cannot create resource "pods" in API group "" in the namespace "test1"
Success! My new user account and RoleBinding are working as expected.
Find the groups an existing Kubernetes user belongs to
You may have created some Kubernetes users in the past and then forgotten what groups you put them in. You can determine what groups a user is in using the following steps:
- Identify your kubeconfig file. The default file is ~/.kube/config, but you may have specified a different file using the KUBECONFIG environmental variable.
- View the contents of your kubeconfig file with an editor of your choice.
- Base64 decode the client-certificate-data value for the user you are interested in.
- One way to do this is to echo the value of the user’s client-certificate-data to base64 -d and redirect the output to a file,
echo \<client-certificate-data value pasted here> | base64 -d > temp-cert.crt
- Check the content of the file that you directed the decoded data to. It should look like this (some content omitted):
-----BEGIN CERTIFICATE----- MIIC8jCCAdqgAwIBAgIIU3QVBMtfet4wDQYJKoZIhvcNAQELBQAwFTETMBEGA1UE ... CONTENT OMITTED ... pUVaAcZxEAbL8JGKgsJrQwBpkfmRM2Dde8ctgLxjD1Dv09eVJdazsrMVgSV2vT0v rrE6/lKCI2tAqlbE1Y9GMDwhJrQHq+fRoUKuN3NYArFkCzVEqEE= -----END CERTIFICATE-----
- Display the certificate data from your temporary certificate file using OpenSSL:
openssl x509 -in temp-cert.crt -text -noout
- Look for the O=some-org entries in the Subject: line of the output. Each of those entries specifies a group the user belongs to.
Hopefully, this blog post helps you with managing users in your clusters you are using for basic purposes such as learning Kubernetes.
If you have questions or feel like you need help with Kubernetes, Docker or anything related to running your applications in containers, get in touch with us at Capstone IT
Docker Accredited Consultant
Certified Kubernetes Administrator
Certified Kubernetes Application Developer