Managing a Kubernetes cluster with one user is easy. Once you go beyond one user, you need to start using Role-Based Access Control (RBAC). I’ve delved into this topic several times in the past with posts on how to Create a Kubernetes User Sandbox in Docker Enterprise and Functional Kubernetes Namespaces in Docker Enterprise. But, once you get beyond a couple of users and/or teams and a few namespaces for them, it quickly becomes difficult to keep track of who can do what and where. And, as time goes on and more and more people have a hand in setting up your RBAC, it can get even more confusing. You can and should have your RBAC resource definitions in source control but it’s not easy to read and is hard to visualize. Enter the open source who-can kubectl plugin from the folks at Aqua Security. It gives you the ability to show who (subjects) can do what (verbs) to what (resources) and where (namespaces).
Install the Krew Plugin Manager
The who-can plugin for kubectl requires the krew plugin manager. If you don’t already have krew installed, the instructions for installing it can be found on its GitHub repository, https://github.com/kubernetes-sigs/krew/. To verify it’s installed correctly, you can get a list of available plugins from kubectl.
$ kubectl plugin list
The following kubectl-compatible plugins are available:
/home/ken.rider/.krew/bin/kubectl-krew
Install the Who-Can Plugin
Once you have the krew plugin manager for kubectl, installing the who-can plugin is easy.
$ kubectl krew install who-can
Setting up RBAC for our Cluster
Now that we have who-can installed, let’s try it out. I have a Kubernetes cluster I built in Azure using Docker Enterprise. There are 3 (non-default) namespaces for development, test, and production. I also have teams for development, test, operations, and security which I’ve created as a cluster admin user. I’ve defined a set of RoleBindings for each along the lines of those I used in my blog post on functional namespaces but taking advantage of the default ClusterRoles created with the Docker Enterprise installation. (See https://github.com/CapKenR/blog/tree/master/who-can for details on the RBAC definition I’m using here.)
$ kubectl apply -f development-namespace.yaml
namespace/development created
rolebinding.rbac.authorization.k8s.io/dev-team:development-edit created
rolebinding.rbac.authorization.k8s.io/test-team:development-view created
rolebinding.rbac.authorization.k8s.io/ops-team:development-admin created
rolebinding.rbac.authorization.k8s.io/sec-team:development-view created
$ kubectl apply -f test-namespace.yaml
namespace/test created
rolebinding.rbac.authorization.k8s.io/dev-team:test-view created
rolebinding.rbac.authorization.k8s.io/test-team:test-edit created
rolebinding.rbac.authorization.k8s.io/ops-team:test-admin created
rolebinding.rbac.authorization.k8s.io/sec-team:test-view created
$ kubectl apply -f production-namespace.yaml
namespace/production created
rolebinding.rbac.authorization.k8s.io/dev-team:production-view created
rolebinding.rbac.authorization.k8s.io/test-team:production-view created
rolebinding.rbac.authorization.k8s.io/ops-team:production-admin created
rolebinding.rbac.authorization.k8s.io/sec-team:production-view created
Take Who-Can for a Spin
Once you have who-can installed, its usage is very straightforward.
$ kubectl kubectl who-can VERB (TYPE | TYPE/NAME | NONRESOURCEURL) [flags]
The typical set of verbs you may be interested in is get
, list
, watch
, create
, update
, patch
, and delete
. There are a few others for specific resource types but, in general, you will be most interested in get
, create
, and delete
.
The typical set of resource types you may be interested in includes pods
, deployments
, services
, persistentvolumeclaims
, configmaps
, secrets
, and more. Also, with the ever-expanding use of CustomResourceDefinitions
, the list of resource types is never-ending.
Who-Can Examples
Let’s first ask “who can get pods in the development namespace?” Notice that all of the teams I created can get (see) pods. There are also quite a few others but most are system service accounts created during the cluster installation. One, tiller, isn’t and we’ll talk about it later.
$ kubectl who-can get pods -n development
ROLEBINDING NAMESPACE SUBJECT TYPE SA-NAMESPACE
dev-team:development-edit development dev-team Group
ops-team:development-admin development ops-team Group
sec-team:development-view development sec-team Group
test-team:development-view development test-team Group
CLUSTERROLEBINDING SUBJECT TYPE SA-NAMESPACE
cluster-admin system:masters Group
compose compose ServiceAccount kube-system
compose-auth-view compose ServiceAccount kube-system
system:controller:clusterrole-aggregation-controller clusterrole-aggregation-controller ServiceAccount kube-system
system:controller:deployment-controller deployment-controller ServiceAccount kube-system
system:controller:endpoint-controller endpoint-controller ServiceAccount kube-system
system:controller:generic-garbage-collector generic-garbage-collector ServiceAccount kube-system
system:controller:namespace-controller namespace-controller ServiceAccount kube-system
system:controller:persistent-volume-binder persistent-volume-binder ServiceAccount kube-system
system:controller:pvc-protection-controller pvc-protection-controller ServiceAccount kube-system
system:controller:statefulset-controller statefulset-controller ServiceAccount kube-system
system:kube-scheduler system:kube-scheduler User
tiller tiller ServiceAccount kube-system
ucp-kube-system:cni-plugin:cluster-admin cni-plugin ServiceAccount kube-system
ucp-kube-system:kube-dns:cluster-admin kube-dns ServiceAccount kube-system
ucp-kube-system:ucp-metrics:cluster-admin ucp-metrics ServiceAccount kube-system
Now let’s ask “who can create pods in the development namespace?” Here we see that only our development and operations teams can create pods there. And the list of system service accounts is shorter but tiller is still there.
$ kubectl who-can create pods -n development
ROLEBINDING NAMESPACE SUBJECT TYPE SA-NAMESPACE
dev-team:development-edit development dev-team Group
ops-team:development-admin development ops-team Group
CLUSTERROLEBINDING SUBJECT TYPE SA-NAMESPACE
cluster-admin system:masters Group
system:controller:clusterrole-aggregation-controller clusterrole-aggregation-controller ServiceAccount kube-system
system:controller:daemon-set-controller daemon-set-controller ServiceAccount kube-system
system:controller:job-controller job-controller ServiceAccount kube-system
system:controller:persistent-volume-binder persistent-volume-binder ServiceAccount kube-system
system:controller:replicaset-controller replicaset-controller ServiceAccount kube-system
system:controller:replication-controller replication-controller ServiceAccount kube-system
system:controller:statefulset-controller statefulset-controller ServiceAccount kube-system
tiller tiller ServiceAccount kube-system
ucp-kube-system:cni-plugin:cluster-admin cni-plugin ServiceAccount kube-system
ucp-kube-system:kube-dns:cluster-admin kube-dns ServiceAccount kube-system
ucp-kube-system:ucp-metrics:cluster-admin ucp-metrics ServiceAccount kube-system
Next, we’ll ask “who can update anything in the development namespace?” Notice that our development can’t just update anything they want in the development namespace with the edit role. For example, they can’t update a RoleBinding
. We don’t want them to be able to change their own permissions. Yet, tiller is there again.
$ kubectl who-can update '*' -n development
No subjects found with permissions to update * assigned through RoleBindings
CLUSTERROLEBINDING SUBJECT TYPE SA-NAMESPACE
cluster-admin system:masters Group
system:controller:clusterrole-aggregation-controller clusterrole-aggregation-controller ServiceAccount kube-system
system:controller:generic-garbage-collector generic-garbage-collector ServiceAccount kube-system
tiller tiller ServiceAccount kube-system
ucp-kube-system:cni-plugin:cluster-admin cni-plugin ServiceAccount kube-system
ucp-kube-system:kube-dns:cluster-admin kube-dns ServiceAccount kube-system
ucp-kube-system:ucp-metrics:cluster-admin ucp-metrics ServiceAccount kube-system
Let’s talk about that tiller service account. The tiller service account is used by Helm to manage resources in a Kubernetes cluster. However, unless you explicitly limit its capabilities, by default the tiller account can do anything with any resource in any namespace. If you need to use helm, you need to implement RBAC for tiller to limit your security exposure. (Or, you could just wait for Helm 3 which doesn’t require tiller.)
Next, let’s look at the other namespaces. “Who can create services in the test namespace?” As you’d expect, only the test and operations teams can.
$ kubectl who-can create services -n test
ROLEBINDING NAMESPACE SUBJECT TYPE SA-NAMESPACE
ops-team:test-admin test ops-team Group
test-team:test-edit test test-team Group
CLUSTERROLEBINDING SUBJECT TYPE SA-NAMESPACE
cluster-admin system:masters Group
system:controller:clusterrole-aggregation-controller clusterrole-aggregation-controller ServiceAccount kube-system
system:controller:persistent-volume-binder persistent-volume-binder ServiceAccount kube-system
tiller tiller ServiceAccount kube-system
ucp-kube-system:cni-plugin:cluster-admin cni-plugin ServiceAccount kube-system
ucp-kube-system:kube-dns:cluster-admin kube-dns ServiceAccount kube-system
ucp-kube-system:ucp-metrics:cluster-admin ucp-metrics ServiceAccount kube-system
Finally, “who can delete services in the production namespace?” And, only the operations team can.
$ kubectl who-can delete services -n production
ROLEBINDING NAMESPACE SUBJECT TYPE SA-NAMESPACE
ops-team:production-admin production ops-team Group
CLUSTERROLEBINDING SUBJECT TYPE SA-NAMESPACE
cluster-admin system:masters Group
system:controller:clusterrole-aggregation-controller clusterrole-aggregation-controller ServiceAccount kube-system
system:controller:generic-garbage-collector generic-garbage-collector ServiceAccount kube-system
system:controller:namespace-controller namespace-controller ServiceAccount kube-system
system:controller:persistent-volume-binder persistent-volume-binder ServiceAccount kube-system
tiller tiller ServiceAccount kube-system
ucp-kube-system:cni-plugin:cluster-admin cni-plugin ServiceAccount kube-system
ucp-kube-system:kube-dns:cluster-admin kube-dns ServiceAccount kube-system
ucp-kube-system:ucp-metrics:cluster-admin ucp-metrics ServiceAccount kube-system
Summary
The who-can kubectl plugin from Aqua Security is a really useful utility for your Kubernetes toolbelt. Setting up role-based access controller for your Kubernetes cluster is extremely important but, also, different for each user. If you want or need help, Capstone IT is a Docker Premier Consulting Partner as well as being an Azure Gold and AWS Select partner. If you are interested in finding out more and getting help with your Container, Cloud and DevOps transformation, please Contact Us.
Ken Rider
Solutions Architect
Capstone IT
Photo by Edo Nugroho on Unsplash