Beginners guide on Kubernetes RBAC with examples

In the previous tutorial we learned about Authentication and Authorization in Kubernetes. With all of the authentication mechanisms we have learned, we need to craft a kubeconfig file that records the details of how we authenticate. kubectl uses this configuration file to determine where and how to issue requests to the API server. This file is typically located in your home directory under ~/.kube/config but may also be specified explicitly on the command line with the --kubeconfig parameter or by way of the KUBECONFIG environment variable.

You can check the tutorial where we installed our multi-node cluster using kubeadm, we had defined KUBECONFIG for root user and the same could be defined for normal user. Before we dive into the details of RBAC in Kubernetes, let us understand about kubeconfig.

Understanding kubeconfig

For someone who may not be familiar with a kubeconfig file, it is important to understand its three top-level structures: users, clusters, and contexts.

With users we name a user and provide the mechanism by which he or she will authenticate to a cluster.
The clusters attribute provides all of the data necessary to connect to a cluster. This, minimally, includes the IP or fully qualified domain name for the API server but may also include items like the CA bundle for a self-signed certificate.
And contexts is where we associate users with clusters as a single named entity. The context serves as the means by which kubectl connects and authenticates to an API server.

Kubernetes Authentication

We know already that Kubernetes supports a number of different authentication providers, including:

HTTP Basic Authentication (largely deprecated)
x509 client certificates
Static token files on the host
Cloud authentication providers like Azure Active Directory and AWS Identity and Access Management (IAM)
Authentication webhooks

In this tutorial we intend to use x508 client certificates to authenticate our user. Kubernetes has no User Objects. User account consists of an authorized certificate that is completed with some authorization as defined in RBAC.

Following are the brief steps required to create a user account:

Create a private/public key pair
Create a Certificate Signing Request
Sign the Certificate
Create kubernetes configuration file that uses these keys to access the cluster
Create an RBAC role
Create an RBAC role binding

Let’s start working on these steps.

Step 1: Create User

First we will create a normal user “user1” which will be part of wheel group. I have already written another detailed tutorial on Linux users and groups

[root@controller ~]# useradd -G wheel user1

Verify the newly created user:

[root@controller ~]# id user1
uid=1004(user1) gid=1004(user1) groups=1004(user1),10(wheel)

Assign a password to this user:

[root@controller ~]# passwd user1

Step 2: Create certificates

You can follow my list of OpenSSL tutorials to understand the basics of generating certificates as here I will be very brief. First we need a private key. I am not using a very strict encryption and it is of just 2048 but size.

[root@controller ~]# openssl genrsa -out user1.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
........................................................+++++
.........................+++++
e is 65537 (0x010001)

Next we create a Certificate Signing Request and it is important that you assign a Subject with the name of the user and the namespace which you plan to use for the user.

[root@controller ~]# openssl req -new -key user1.key -out user1.csr -subj "/CN=user1/O=dev"

Lastly create the certificate

[root@controller ~]# openssl x509 -req -in user1.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out user1.crt -days 365
Signature ok
subject=C = IN, ST = KARNATAKA, L = BENGALURU, O = GOLINUXCLOUD, OU = TEST, CN = controller, emailAddress = admin@golinuxcloud.com
Getting CA Private Key

Step 3: Create namespace (optional)

If you plan to use any existing namespace then you can ignore this step but I want to use a separate namespace to implement the RBAC policy so I will create one.

[root@controller ~]# kubectl create namespace dev
namespace/dev created

Step 4: Update Kubernetes Config file with User Credentials

Since we are using x509 certificates for authentcating the user, the same needs to be added in the kubernetes config file. Following is the default kubeconfig file:

[root@controller ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.43.48:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

Use the subcommand set-credentials as kubectl config set-credentials <CREDENTIAL_NAME> to add a credential into kubeconfig

[root@controller ~]# kubectl config set-credentials user1 --client-certificate=user1.crt --client-key=user1.key
User "user1" set.

Next verify the kubernetes config file, it should also contain the credentials of the newly added user:

[root@controller ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.43.48:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
- name: user1
  user:
    client-certificate: /root/user1.crt
    client-key: /root/user1.key

Step 5: Create security context for new user

A context defines which user to use with which cluster, but can also define the namespace that kubectl should use, when you don’t specify the namespace explicitly with the --namespace or -n option.

The following command is used to create a new context user1-context that ties together the cluster and the user you created:

[root@controller ~]# kubectl config set-context user1-context --cluster=kubernetes --namespace=dev --user=user1
Context "user1-context" created.

HINT

You can use kubectl config get-clusters to get the cluster name

To list the available contexts:

[root@controller ~]# kubectl config get-contexts
CURRENT   NAME                          CLUSTER      AUTHINFO           NAMESPACE
*         kubernetes-admin@kubernetes   kubernetes   kubernetes-admin
          user1-context                 kubernetes   user1              dev

Next verify if you permission to list pods under this context:

[root@controller ~]# kubectl --context=user1-context get pods
Error from server (Forbidden): pods is forbidden: User "user1" cannot list resource "pods" in API group "" in the namespace "dev"

As expected user1 is not allowed to access user1-context at the moment as we have not yet assigned a role and rolebinding to this user.

A role is a set of abstract capabilities. For example, the appdev role might represent the ability to create Pods and services.
A role binding is an assignment of a role to one or more identities. Thus, binding the appdev role to the user identity alice indicates that Alice has the ability to create Pods and services.

Example-1: Configure RBAC to define new role with “modify” permission

Create new role

We will create a new role “dev” based on the namespace where we want to apply this role. You can also choose any other name for the role. To get the KIND and apiVersion for Role we can explore api-resources:

[root@controller ~]# kubectl api-resources | grep -iE 'role|KIND'
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
clusterrolebindings                            rbac.authorization.k8s.io      false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io      false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io      true         RoleBinding
roles                                          rbac.authorization.k8s.io      true         Role

HINT

In Kubernetes there are two pairs of related resources that represent roles and role bindings. One pair applies to just a namespace (Role and RoleBinding) while the other pair applies across the cluster (ClusterRole and ClusterRoleBinding).

Here we are interested in KIND Role, next let’s get the apiVersion for this resource:

[root@controller ~]# kubectl explain Role | head -n 2
KIND:     Role
VERSION:  rbac.authorization.k8s.io/v1

So now we have both KIND and apiVersion of Role. Following is my YAML file to create dev role that gives an identity the ability to create and modify Pods, Deployments and ReplicaSets and can be applied to any of the three types of API Groups:

[root@controller ~]# cat dev-role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
   namespace: dev
   name: dev
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["deployments", "replicasets", "pods"]
  verbs: ["list", "get", "watch", "create", "update", "patch", "delete"]

Understanding Verbs for Kubernetes roles

Roles are defined in terms of both a resource (e.g., “Pods”) and a verb that describes an action that can be performed on that resource. The verbs correspond roughly to HTTP methods. The commonly used verbs in Kubernetes RBAC are listed in the following table:

Verb	HTTP Method	Description
create	POST	Create a new resource.
delete	DELETE	Delete an existing resource.
get	GET	Get a resource.
list	LIST	List a collection of resources.
patch	PATCH	Modify an existing resource via a partial change.
update	PUT	Modify an existing resource via a complete object.
watch	GET	Watch for streaming updates to a resource.
proxy	GET	Connect to resource via a streaming WebSocket proxy.

We will create this role:

[root@controller ~]# kubectl create -f dev-role.yml
role.rbac.authorization.k8s.io/dev created

Bind a user to the newly created role

Next we create RoleBinding to bind this Role to the user1. Following is our YAML file:

[root@controller ~]# cat user1-rolebind.yml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
   name: dev-role-binding
   namespace: dev
subjects:
- kind: User
  name: user1
  apiGroup: ""
roleRef:
  kind: Role
  name: dev
  apiGroup: ""

Here we have deined the user to which this rolebinding must be applied to as user1.

Let’s go ahead and create this RoleBinding:

[root@controller ~]# kubectl create -f user1-rolebind.yml
rolebinding.rbac.authorization.k8s.io/dev-role-binding created

Verify the RBAC Policy

Now that we have created a role with permission to create and modify different resources for user1, let’s verify the same:

[root@controller ~]# kubectl --context=user1-context get pods
No resources found in dev namespace.

If you recall this command was throwing a “FORBIDDEN” error previously before applying the role and role binding but now we get a proper message so it means the RBAC is working as expected.

We can describe the role to get more information on the Verbs which are allowed for different resources:

[root@controller ~]# kubectl -n dev describe role dev
Name:         dev
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  deployments             []                 []              [list get watch create update patch delete]
  pods                    []                 []              [list get watch create update patch delete]
  replicasets             []                 []              [list get watch create update patch delete]
  deployments.apps        []                 []              [list get watch create update patch delete]
  pods.apps               []                 []              [list get watch create update patch delete]
  replicasets.apps        []                 []              [list get watch create update patch delete]
  deployments.extensions  []                 []              [list get watch create update patch delete]
  pods.extensions         []                 []              [list get watch create update patch delete]
  replicasets.extensions  []                 []              [list get watch create update patch delete]

Let’s try to create a new deployment resource under user1-context to verify if user1 has enough permission:

[root@controller ~]# kubectl create deployment devnginx --image=nginx --context=user1-context
deployment.apps/devnginx created

As expected the deployment was successfully created but we can’t find it under default namespace:

[root@controller ~]# kubectl get pods
NAME             READY   STATUS    RESTARTS   AGE
nginx-cm         1/1     Running   2          6d11h
nginx-dev        1/1     Running   1          6d6h
secret-tls-pod   1/1     Running   2          6d6h

That’s because user1-context is applied for dev namespace so by default any resource would be created under dev namespace only:

[root@controller ~]# kubectl get pods -n dev
NAME                        READY   STATUS              RESTARTS   AGE
devnginx-7bc9b989dc-v6zs9   0/1     ContainerCreating   0          8s

Testing Authorization with can-i

The first useful tool is the auth can-i command for kubectl. This tool is very useful for testing if a particular user can do a particular action. You can use can-i to validate configuration settings as you configure your cluster, or you can ask users to use the tool to validate their access when filing errors or bug reports.

In its simplest usage, the can-i command takes a verb and a resource.

[root@controller ~]# kubectl auth can-i create pods --context=user1-context
yes

[root@controller ~]# kubectl auth can-i create service --context=user1-context
no

Since we have given permission to create Pods, Deployments and ReplicaSets only, the auth can-i gives a “no” for creating a service

Example-2: Configure RBAC to define new role with “view-only” permission

In this example we will create another role which will have permission to only view the available resources but not to create or modify. Let me create a new role as “view-only” with following content:

[root@controller ~]# cat view-only-role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
   namespace: view-only
   name: view-only
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["deployments", "replicasets", "pods"]
  verbs: ["list", "get", "watch"]

As you see I have given Verbs as list, get and watch which would only give viewing permission to the allowed resources part of the API group as mentioned above. I will apply these changed into a new namespace “view-only”

Now before we create this Role, we must make sure the provided namespace i.e. “view-only” exists, so let me go ahead and create one for our demonstration:

[root@controller ~]# kubectl create namespace view-only
namespace/view-only created

Next I will create this role:

[root@controller ~]# kubectl create -f view-only-role.yml
role.rbac.authorization.k8s.io/view-only created

Now we need to create a RoleBinding to bind this role to our user1

[root@controller ~]# cat view-only-rolebinding.yml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
   name: view-only-role-binding
   namespace: view-only
subjects:
  - kind: User
    name: user1
    apiGroup: ""
roleRef:
  kind: Role
  name: view-only
  apiGroup: ""

Let’s create this RoleBinding:

[root@controller ~]# kubectl create -f view-only-rolebinding.yml
rolebinding.rbac.authorization.k8s.io/view-only-role-binding created

We are all set to create a new security context “viewonly-context” which can be used to give view only permission to user1

[root@controller ~]# kubectl config set-context viewonly-context --cluster=kubernetes --namespace=view-only --user=user1
Context "viewonly-context" created.

List the available contexts in the kubeconfig file:

[root@controller ~]# kubectl config get-contexts
CURRENT   NAME                          CLUSTER      AUTHINFO           NAMESPACE
*         kubernetes-admin@kubernetes   kubernetes   kubernetes-admin
          user1-context                 kubernetes   user1              dev
          viewonly-context              kubernetes   user1              view-only

Next we can verify our viewonly-context by trying to list the available pods.

[root@controller ~]# kubectl --context=viewonly-context get pods
No resources found in view-only namespace.

Since the context has permission to list the resources, the command has executed successfully. We can get more details on the available verbs for this role using kubectl describe:

[root@controller ~]# kubectl -n view-only describe role view-only
Name:         view-only
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources               Non-Resource URLs  Resource Names  Verbs
  ---------               -----------------  --------------  -----
  deployments             []                 []              [list get watch]
  pods                    []                 []              [list get watch]
  replicasets             []                 []              [list get watch]
  deployments.apps        []                 []              [list get watch]
  pods.apps               []                 []              [list get watch]
  replicasets.apps        []                 []              [list get watch]
  deployments.extensions  []                 []              [list get watch]
  pods.extensions         []                 []              [list get watch]
  replicasets.extensions  []                 []              [list get watch]

Next let’s try to create a new Deployment using this context:

[root@controller ~]# kubectl create deployment testnginx --image=nginx --context=viewonly-context
error: failed to create deployment: deployments.apps is forbidden: User "user1" cannot create resource "deployments" in API group "apps" in the namespace "view-only"

So this has failed with FORBIDDEN error as we had not given permission to create new resources.

We can also use auth can-i command to verify the user’s privilege under viewonly-context:

[root@controller ~]# kubectl auth can-i create pods --context=viewonly-context
no

[root@controller ~]# kubectl auth can-i get pods --context=viewonly-context
yes

[root@controller ~]# kubectl auth can-i get service --context=viewonly-context
no

So this context has privilege to list pods but has no privilege to create a pod or list service resource objects.

Deleting contexts

To clean up the list of contexts, you can either delete the entries from the kubeconfig file manually or use the following two commands:

[root@controller ~]# kubectl config delete-context user1-context
deleted context user1-context from /etc/kubernetes/admin.conf

[root@controller ~]# kubectl config delete-context viewonly-context
deleted context viewonly-context from /etc/kubernetes/admin.conf

Deleting Role

Use the following command to delete a role along with the name of the namespace:

[root@controller ~]# kubectl delete role dev -n dev
role.rbac.authorization.k8s.io "dev" deleted

Deleting RoleBinding

Use the following command to delete a role binding along with the namespace under which the binding was created:

[root@controller ~]# kubectl delete rolebinding dev-role-binding -n dev
rolebinding.rbac.authorization.k8s.io "dev-role-binding" deleted

Conclusion

When you begin with a small cluster and a small team, it is sufficient to have every member of the team have equivalent access to the cluster. But as teams grow and products become more mission critical, limiting access to parts of the cluster is crucial. In a well-designed cluster, access is limited to the minimal set of people and capabilities needed to efficiently manage the applications in the cluster. Understanding how Kubernetes implements RBAC and how those capabilities can be used to control access to your cluster is important for both developers and cluster administrators.