Securing Kubernetes Clusters with Kyverno: A Step-by-Step Guide

In my previous blog, I discussed the importance of policy management in Kubernetes and introduced Open Policy Agent (OPA) as a powerful tool for implementing policy controls. Today, I'd like to expand on that discussion by introducing another open-source tool, Kyverno, which provides a more efficient way to write policies using YAML.

As we saw in the last blog, policies are essential for ensuring security, compliance, and governance in Kubernetes environments. However, writing policies can be complex and require expertise in languages like Rego. Kyverno simplifies this process by allowing users to write policies in YAML, making it more accessible to a wider range of users.

In this blog, I'll explore how Kyverno can help us write policies more efficiently and effectively. I'll also compare Kyverno with OPA and discuss the benefits of using both tools together.

Key differences:

• Kyverno uses YAML policies, while OPA uses Rego

• Kyverno focuses on Kubernetes resources and image verification, while OPA supports multiple environments and advanced logic

• Kyverno provides self-service reports and policy exceptions, while OPA requires separate audit logs and exception handling

In this blog, we'll explore how to write validating policies using Kyverno.(Note that while Kyverno is a powerful tool for policy management, it's not covered in the Certified Kubernetes Security (CKS) exam. For the CKS exam, you'll need to focus on learning OPA.)Let's dive into Kyverno's validating policies!

Let's install Kyverno in our cluster using the following command:

kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.12.0/install.yaml

It will create a bunch of resources in new namespace "kyverno"

This will create a bunch of resources in a new namespace called "kyverno". We'll use the same three examples to explore validating policies using Kyverno.

Scenario: Ensuring all containers run with non-root users for enhanced security.

Create new clusterpolicy.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: k8sdisallowrootuser
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-either-container-using-rootuser
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message:
          "Running as root is not allowed. The fields spec.securityContext.runAsNonRoot,
          spec.containers[*].securityContext.runAsNonRoot, and
          spec.initContainers[*].securityContext.runAsNonRoot must be `true`."
        anyPattern:
          - spec:
              securityContext:
                runAsNonRoot: true
              containers:
                - =(securityContext):
                    =(runAsNonRoot): true
              =(initContainers):
                - =(securityContext):
                    =(runAsNonRoot): true

          - spec:
              containers:
                - securityContext:
                    runAsNonRoot: true
              =(initContainers):
                - securityContext:
                    runAsNonRoot: true

Now apply that policy.

Test a Violating Pod

Now, let's test a pod that violates the policy:

Now Test a Compliant Pod

Finally, let's test a pod that complies with the policy:

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: complaint-pod
  name: complaint-pod
spec:
  containers:
    - image: nginx
      name: complaint-pod
      securityContext:
        runAsNonRoot: true

2. Scenario: Enforcing resource limits and requests to ensure fair usage and prevent resource exhaustion.

Create new clusterpolicy.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: all-containers-need-requests-and-limits
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-container-resources
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "All containers must have CPU and memory resource requests and limits defined."
      pattern:
        spec:
          containers:
          - name: "*"
            resources:
              limits:
                memory: "?*"
                cpu: "?*"
              requests:
                memory: "?*"
                cpu: "?*"

Now Test a Violating Pod

Now Test a Compliant Pod

apiVersion: v1
kind: Pod
metadata:
  name: compliant-pod
spec:
  containers:
    - name: my-container
      image: nginx
      securityContext:
        runAsNonRoot: true
      resources:
        limits:
          memory: "128Mi"
          cpu: "500m"
        requests:
          memory: "64Mi"
          cpu: "250m"

3. Scenario: If you want to whitelist image to ensure only approved container images are deployed.

create cluster policy

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-image-registries
spec:
  validationFailureAction: Enforce
  rules:
    - name: validate-registries
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Images may only come from our internal enterprise registry."
        pattern:
          spec:
            containers:
              - image: "docker.io/*"

Now Test a Violating Pod

Now Test a Compliant Pod

That's all for this blog post! If you're interested in learning more, I encourage you to check out Kyverno's awesome documentation and give it a try. I've found it to be a game-changer for securing my cluster in a more efficient way. In my next blog post, we'll explore another exciting aspect of Kyverno: "Mutating". Stay tuned! Until then, keep on hustling!