James Alvarez

Kubernetes stands as the supreme choice among container orchestration systems, captivating innumerable enterprises that are constructing and transitioning their operations to this widely embraced open-source technology. A meticulous investigation conducted by the Cloud Native Computing Foundation during the year 2021 uncovered that an astonishing 96% of the participants engaged in the survey had either implemented or extensively assessed the prospect of integrating Kubernetes into their infrastructure.

Nonetheless, the escalated employment of this technology necessitates vigilant teams to confront and mitigate each potential security hazard that might jeopardize the multifaceted strata and functionalities existing within a Kubernetes cluster.

Enter Kube-hunter, a remarkable open-source tool in the realm of Kubernetes security, under the astute management of Aqua Security. This innovative solution empowers administrators, operators, and security contingents to systematically uncover vulnerabilities nestled within their deployments, proactively rectifying these concerns long before malevolent actors can exploit any fissures in the system’s armor.

Exploring the Mechanics of Kube-hunter

Kube-hunter’s Core Objective

Kube-hunter is specifically designed to identify vulnerabilities within Kubernetes infrastructures. By scanning designated domains or address ranges, it seeks out ports that are related to Kubernetes. These ports, if improperly configured, can potentially act as gateways for security threats.

Key Functions of Kube-hunter:

  • Detects open Kubernetes-related ports;
  • Assesses configuration setups to ensure they are secure;
  • Flags vulnerabilities, helping administrators take preventive measures.

The Testing Procedure

Kube-hunter follows a systematic approach to pinpoint weaknesses in a Kubernetes setup:

  • Initial Scanning: Kube-hunter begins by probing the given domain or address range to find any open ports related to Kubernetes;
  • In-depth Verification: After identifying potential points of entry, kube-hunter then carries out a series of tests. These tests are meticulously crafted to mimic the strategies attackers might use to exploit vulnerabilities.

Two Modes of Operation: Passive and Active Hunting

While Kube-hunter is built to be non-intrusive, it offers two modes of operation to cater to different security needs:

  • Passive Hunting (Default Mode): In this mode, kube-hunter conducts a non-disruptive examination of the system. No changes are made to the state of the cluster, ensuring the stability and integrity of the Kubernetes setup. This mode is highly recommended for most users, especially those who are only seeking to identify vulnerabilities without affecting operations;
  • Active Hunting: This mode delves deeper, simulating actions that a malicious actor might take to exploit vulnerabilities. It’s important to note that while active hunting provides a comprehensive understanding of potential threats, it might alter the cluster’s state. Consequently, users should approach this mode with caution and ideally run it in a controlled environment.

Recommendations for Safe Usage:

  • Always begin with passive hunting to get a basic overview of potential vulnerabilities;
  • If considering active hunting, conduct this in a test or isolated environment to avoid potential disruptions;
  • Stay updated with best practices and recommendations from Kubernetes communities for securing clusters.

Ensuring Cluster Safety

Understanding what a hacker might be able to do is a vital step in securing your cluster. By highlighting potential risks, Kube-hunter aids organizations in reinforcing their defenses, ensuring that their Kubernetes clusters remain robust against external threats.

Insights for Enhanced Security:

  • Regularly scan the cluster with tools like Kube-hunter;
  • Educate the IT team about the importance of proper Kubernetes port configurations;
  • Collaborate with security experts to establish a resilient Kubernetes infrastructure.

Kube-hunter Tests Overview

Kube-hunter is an advanced tool that tests Kubernetes clusters for vulnerabilities. With a diverse range of tests, Kube-hunter actively seeks potential weak points in your cluster configuration, ensuring that it remains secure. The tests can be broadly classified into ‘Passive Hunters’ and ‘Active Hunters’. Here’s a detailed breakdown:

Passive Hunters:

These tests check for potential vulnerabilities without making any alterations to the system:

API Service Discovery:

  • Searches for Kubernetes API Services existence;
  • Essential for understanding exposure levels of the cluster.

K8s Dashboard Discovery:

  • Identifies if a Dashboard is present;
  • Monitoring tools can sometimes have misconfigurations.

Etcd Service:

  • Searches for etcd service existence;
  • Ensures that etcd, the primary key-value store for K8s, is properly secured.

Host Discovery:

  • Generates IP addresses to inspect, based on specified cluster/scan types;
  • Determines how the system identifies and interacts with hosts.

Kubectl Client Discovery:

  • Investigates the presence of a local kubectl client;
  • Evaluates the tools available to users and administrators.

Kubelet Discovery:

  • Searches for a Kubelet service and examines its open ports;
  • Evaluates how node agents are configured and communicated with.

Port Scanning:

  • Scans well-known Kubernetes ports for open endpoints;
  • Useful for determining potentially vulnerable communication ports.

Proxy Discovery:

  • Identifies if an open Proxy service is present;
  • Proxies can often have their vulnerabilities if not set up securely.

Kubelet ReadOnly and Secure Ports Hunters:

Scrutinizes specific endpoints on open ports of both readonly and secure Kubelet servers.

Ensures that crucial Kubelet ports are shielded from unauthorized access.

Azure AKS Hunting:

  • Focuses on hunting specific configurations in Azure cluster deployments;
  • Azure-specific configurations can sometimes be overlooked.

API Server Exploration:

  • Checks the accessibility of the API server;
  • Confirms if malicious agents can tap into the server using tokens from compromised pods.

API Version Analysis:

  • Aims to fetch the API Server’s version from its /version endpoint;
  • Older versions can be prone to vulnerabilities.

Pod Capabilities:

  • Verifies if there are default enabled capabilities in a pod;
  • Excessive capabilities can elevate privileges.

Certificate Email Scrutiny:

  • Inspects Kubernetes SSL certificates for email addresses;
  • Reduces chances of information leakage.

K8s & Kubectl CVE Check:

  • Assesses if the system is vulnerable to notable CVEs;
  • Keeps track of known vulnerabilities.

Dashboard & Proxy Analysis:

  • Hunts for open dashboards and proxies;
  • Gathers information on cluster node types and configurations.

Etcd Remote Access:

  • Inspects the etcd for remote availability, version, and read access;
  • Critical for ensuring database security.

Mount Examination – /var/log:

Seeks pods with write access to host’s /var/log. If found, it implies potential unauthorized access to host files.

Secrets Access Analysis:

  • Determines the secrets that can be accessed by the pod;
  • Secrets leakage can be catastrophic for cluster security.

Active Hunters:

These tests might attempt to exploit vulnerabilities:

Secure Kubelet Port Misuse:

  • Tries to exploit misconfigurations in kubelet’s secure port, potentially granting attackers entry or privilege escalation.

Kubelet Run, Logs & System Analysis:

  • Executes commands inside containers or fetches logs;
  • Reveals potential information leakage or system misconfigurations.

Azure SPN Exploration:

  • Seeks the Azure subscription file inside a container;
  • Checks for potential Azure cloud vulnerabilities.

ARP & DNS Spoofing Analysis:

  • Evaluates the feasibility of ARP or DNS spoof attacks from within a pod;
  • Essential for ensuring internal network security.

Etcd Remote Write Check:

  • Attempts to add a new key to the etcd DB to check for write access;
  • Confirms the security level of database operations.

Prove /var/log Mount Exploration:

  • Tries accessing sensitive host files from a mounted pod;
  • Evaluates the risk of potential data breaches.

Build Date & K8s Version Analysis:

  • If a proxy is exposed, extracts crucial data like build date or Kubernetes version;
  • Helps in assessing version-specific vulnerabilities.

Recommendations:

  • Always run Passive Hunters before diving into Active Hunters to ensure the safety and stability of your systems;
  • Keep a backup of your cluster configuration before running any tests;
  • Use Kube-hunter alongside other security tools to ensure comprehensive coverage;
  • Regularly update and patch your Kubernetes clusters to minimize vulnerabilities;
  • Use the insights from these tests to educate your team on best practices.

By understanding and employing the tests Kube-hunter offers, organizations can ensure their Kubernetes deployments remain resilient against potential attacks.

Kube-hunter Installation & Execution Guide

Kube-hunter is an essential tool to test Kubernetes clusters’ security. Powered by Aqua Security, it offers various modes of scanning to uncover potential vulnerabilities. Below is a comprehensive guide to install and run Kube-hunter effectively:

A man is working on a laptop in the office

1. Command Line Installation

To use Kube-hunter from the command line, follow these steps:

Pre-requisites:

Ensure that Python 3 and pip are already installed on the machine.

Installation:

To install Kube-hunter, execute:

pip install kube-hunter

Running Kube-hunter:

  • Upon executing Kube-hunter, users will be given a choice among three scanning options;
  • Remote scanning: This mode targets specific IP addresses or DNS names;
  • Interface scanning: Scans subnets across all local network interfaces;
  • IP range scanning: Targets a predefined range of IP addresses.

Result Dispatching:

By default, Kube-hunter displays results directly on the screen (stdout). However, for more flexibility:

Dispatch results via HTTP by setting these environment variables:

  • KUBEHUNTER_HTTP_DISPATCH_URL: The default value is https://localhost;
  • KUBEHUNTER_HTTP_DISPATCH_METHOD: By default, it’s set to POST.

2. Docker Container Execution

For those who prefer containerized applications, Kube-hunter can be executed within a Docker container:

Running Kube-hunter in Docker:

Execute the following command to initiate the Kube-hunter container:

docker run -it –rm –network host aquasec/kube-hunter

The command initializes a Docker container, and the CLI is run within this environment.

Accessing Results:

Post execution, a unique public URL (associated with a provided token) will appear to review the scan results.

Example URL:

https://kube-hunter.aquasec.com/report.html?token=ABC123.ABC123

Note: The sample URL shown is based on a test performed on a Minikube cluster.

3. Kube-hunter Pod Execution

In scenarios where the external IP of nodes isn’t directly reachable, Kube-hunter can be run as a pod inside the Kubernetes cluster:

Pod Configuration:

Here’s a basic Kubernetes job definition to run Kube-hunter as a pod:

apiVersion: batch/v1

kind: Job

metadata:

  name: kube-hunter

spec:

  template:

    spec:

      containers:

        – name: kube-hunter

          image: aquasec/kube-hunter

          command: [“kube-hunter”]

          args: [“–pod”]

      restartPolicy: Never

  backoffLimit: 4

Recommendations:

  • Ensure to monitor the job’s progress and retrieve results once it completes;
  • Understand that running Kube-hunter directly within the cluster may have implications, so always run in controlled environments and be aware of the potential impact;
  • Final Thoughts: Kube-hunter provides versatile ways to assess Kubernetes cluster security. Whether you opt for a CLI, Docker container, or pod approach, regular security checks with tools like Kube-hunter are instrumental in maintaining a secure Kubernetes environment.

Conclusion

Kube-hunter stands as an exceptional tool for conducting penetration tests on Kubernetes environments. Its remarkable functionality allows for deployment on a variety of systems, even on your personal laptop. By opting for Remote scanning and furnishing the pertinent IP address or domain name of your Kubernetes cluster, you promptly unveil a perspective akin to that of a potential attacker, gazing critically upon your intricate Kubernetes configuration. Given the prevalent integration and dependence on Kubernetes across diverse domains, we assert the utmost importance of safeguarding your cluster against any lurking security vulnerabilities.