In this article I'm going to talk about some security considerations that you need to implement in order to enhance the security of your Kubernetes cluster. The scope of this post is gonna focus on those 3 layers:

Kubernetes core components

Encrypting data at rest

Kubernetes uses Etcd to store all the cluster data. To encrypt the data you should start the apiserver with the flag --encryption-provider-config=/path/to/config/file This is an example of a config file

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - identity: {}
      - aescbc:
          keys:
            - name: key1
              secret: {{ encryption_key | b64encode }}

the field resources lists the API objects to encrypt, in this example only the secrets are encrypted.

The encryption key can be provided in the config file or by a KMS. To learn more check this link

Secure the communications with TLS

Enable TLS communication between all the components. You need to provision a PKI for your cluster (Take a look at the post k8s-the-hard-way to learn how to do it with Ansible).

Starting from Kubernetes 1.4, a new API object called certificates.k8s.io was introduced to manage automatically the generation of certificates for Node components (kublet and kube-proxy). Check this link to learn more.

Enable Admission control plugins

These are some plugins that enhance the security of the k8s cluster.

• NodeRestriction : Limit kubelet power.
• PodSecurityPolicy : Define a security context for a pod.
• EventRateLimit : Limits the number of requests to the apiserver.

The full list of admission plugins here

I would recommend to start the apiserver at least with these of plugins:

--enable-admission-plugins=NamespaceLifecycle,\
                            NodeRestriction,\
                            LimitRanger,\
                            ServiceAccount,\
                            DefaultStorageClass,\
                            ResourceQuota,\
                            PodSecurity

You may want to add the flag --allow-privileged=true to the apiserver otherwise root containers wont' be able to run.

Enable Mandatory Access Control

enable RBAC by starting the apiserver with --authorization-mode=RBAC ...

Disable anonymous requests

Start the apiserver with --anonymous-auth=false flag. for kubelet, the config file should contains the authentication part like this one

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
authentication:
  anonymous:
    enabled: false

Auditing

These are some important tools to use to audit your cluster:

• kube-bench: Checks that the cluster is deployed according to the best practices.
• falco: Checks for abnormalities and intrusions by analyzing the activities in the cluster.

Container runtime

Secure images

Always scan the images for vulnerabilities. These are some tools that can help you with that:

• Grype
• Trivy
• Clair

To build a chain of trust for your images you may want to take a look at these tools:

• Notary
• Portieris
• harbor is a fantastic cloud repository that integrates some of the tools mentioned above.

Secure Runc

Use gVisor to ensure a better isolation between the application and the host kernel To use gVisor in Kubernetes, you'll need to setup a Runtime-Class.

Pods

Network policies

Use a network plugin that supports Network policy like Calico.

Service mesh

Deploy a service mesh like istio to secure the communications between the pods.

Policy controller

Use gatekeeper to enforce policies in your cluster.

Audit

These are some useful tools for auditing:

• kubesec: Analyze your Kubernetes resources for security flaws.
• kubeaudit: Audit your Kubernetes clusters against common security controls.

If you think about some useful tool that I forgot to mention please let me know and I'll be glad to add it !