Kubernetes has made an enormous impact on the landscape of software development. It furnishes a robust framework for deploying applications, effectively shielding you from the intricate intricacies of the underlying infrastructure. However, akin to many remarkable solutions, it entails a trade-off. The ease and seamlessness it brings to deployments can inadvertently lead to excessive allocation of resources, potentially inflating cloud expenditures.
Introducing Kubecost, a tool designed to provide real-time visibility into your resource consumption. It furnishes a comprehensive breakdown of your expenses, enabling you to gain a clear understanding of your cost structure. Kubecost also empowers you to allocate expenses outside the cluster (such as database or storage costs), facilitating a holistic perspective of your cloud outlays. Additionally, it facilitates the configuration of notifications to promptly identify spikes in costs and take appropriate measures in response.
What is Kubecost?
Kubecost represents a dynamic open-source solution designed to enhance cost monitoring and management. This versatile tool caters not only to software development teams but also to a wide range of professionals seeking to gain valuable insights into various aspects of cost management, such as allocation, tracking, and alerts. With a strong focus on Kubernetes and Cloud-native applications, Kubecost offers a comprehensive approach to addressing financial considerations within these technological landscapes.
Kubecost’s Open Source Paradigm: Exploring Its Core
Kubecost follows an open core approach, where the commercial product is constructed upon the foundation of the Kubecost open source project, accessible on GitHub. Notably, it can be effortlessly deployed using a single Helm installation. Essentially, Kubecost becomes an integral part of an organization’s infrastructure, enabling real-time cost monitoring. An additional advantage is that data remains within the company’s confines, eliminating the necessity of remote data sharing.
The Role of Kubernetes
Kubernetes stands as an open-source powerhouse, orchestrating container operations with finesse. By automating the intricate tasks tied to deploying and scaling containerized applications, it simplifies the landscape of management, presenting a realm of efficiency and ease.
However, the potential of Kubernetes and containers extends far beyond this. Enterprises stand to reap immense rewards across the domains of DevOps, application development, and delivery. These technologies serve as catalysts for transformative possibilities, such as breathing new life into traditional applications through modernization efforts. Furthermore, they act as enablers for seamless hybrid and multi-cloud strategies, fostering adaptability and choice.
One of their standout contributions is accelerating the pace of cloud-native application development. The agility and speed inherent in this approach empower businesses to swiftly address market demands and evolving requirements. In essence, Kubernetes and containers coalesce into a force that reshapes the paradigms of IT, empowering businesses to navigate the digital landscape with innovation and confidence.
Installing Kubecost: A Comprehensive Guide
In the year 2018, the Kubecost development team embarked on a mission to establish an effective means of monitoring Kubernetes resources. Leveraging Prometheus and Grafana, they sought to offer insights into cost allocation, thus enhancing visibility within the Kubernetes environment.
Their journey, however, encountered swift challenges. The Grafana dashboards they initially employed fell short in several key aspects. These limitations included the inability to seamlessly display cost allocation categorized by service, deployment, or label. Moreover, these dashboards struggled to integrate with dynamic cloud billing data, impeding their ability to provide a comprehensive cost overview. Additionally, they grappled with the challenge of accommodating cross-cluster data visualization.
As the year 2019 dawned, the team took a pivotal step. They introduced Kubecost, an open-source solution designed to generate and present cost allocation data linked to Kubernetes clusters. This marked a significant leap forward in overcoming the hurdles they had encountered, promising a more streamlined and accurate approach to resource monitoring and cost analysis.
Key features offered by Kubecost
- Real-time Cost Breakdown: Provides immediate insights into cost distributions across essential Kubernetes entities, encompassing namespaces, deployments, services, daemon sets, pods, containers, jobs, and more;
- Customizable Label-Based Allocation: Enables the measurement of expenditures based on user-defined labels, affording visibility into spending attributed to owners, teams, departments, products, and other categories;
- Dynamic Pricing Integration: Seamlessly integrates with AWS and GCP billing APIs, allowing for on-the-fly asset pricing estimations. This functionality extends to Azure as well;
- Resource-Centric Cost Metrics: Offers granular cost allocation metrics covering CPU, GPU, memory, and storage usage;
- External Cloud Cost Attribution: Links out-of-cluster cloud costs, such as those tied to S3 buckets and RDS instances, back to the relevant owner entities, like pods or deployments;
- Billing Data Export: Facilitates the export of billing data to Prometheus, enabling in-depth analysis and further exploration.
However, it’s essential to note the limitations of the free tier:
| Feature | Free Tier Limitation |
|---|---|
| Single Cluster Limitation | Supports only one cluster; multi-cluster setups not available |
| 15-Day Metric Retention | Metric data retention restricted to 15 days |
Requirements to Set Up Kubecost
The default configuration of the Kubecost Helm chart incorporates both Prometheus and Grafana instances, offering notable advantages especially if you lack these components in your setup. Moreover, the package includes useful dashboards that provide the capability to monitor essential metrics, such as pod and node utilization, effectively enhancing your monitoring capabilities.
Installing Kubecost
To begin with, establish a namespace dedicated to Kubecost:
apiVersion: v1
kind: Namespace
metadata:
name: kubecost
In the following section, we present the Helm values that will be employed in our installation procedure.
values:
networkCosts:
enabled: true
kubecostProductConfigs:
clusterName: my-k8-cluster
projectID: florian-test-123456
bigQueryBillingDataDataset: all_billing_data.gcp_billing_export_v1_XXXXXX_XXXXXX_XXXXXX
serviceKeySecretName: gcp-secret
global:
thanos:
enabled: false
Accessing Kubecost’s Web User Interface (WebUI)
For a rapid and secure means of reaching the WebUI offered by Kubecost, you can enable port-forwarding by executing the following command:
kubectl port-forward –namespace kubecost deployment/kubecost-cost-analyzer 9090:9090
Configuring Kubecost Cloud Integration for GCP
Kubecost’s automated system identifies your cloud provider and retrieves pricing information for in-cluster assets from Azure, AWS, and GCP.
- Once you’ve successfully set up the external cloud cost configuration with your cloud provider, you’ll gain the capability to juxtapose Kubernetes cost metrics alongside the expenses of external cloud services like S3 or BigQuery. This integration also enables you to factor in enterprise discounts, spot market prices, commitment discounts, and more, ensuring a comprehensive and accurate cost overview;
- To establish Kubecost with GCP, follow these steps: Go to the settings menu and locate “External Cloud Cost Configuration (GCP).” Click on the “Update” button to initiate the configuration process;
- Remember to generate a secret using your service key:
kubectl create secret generic gcp-secret –from-file compute-viewer-kubecost-key.json
- Please be aware that when using the free version of Kubecost, it will not factor in any potential discounts associated with your GCP account.
Kubecost’s Key Functionalities
Overview
At the heart of Kubecost lies the Overview dashboard, featured as the default landing page. This dashboard comprises a set of immovable panels, each with its distinct purpose, as illustrated below:
Cost Allocation
Assets
Savings
This segment also provides a platform for presenting preliminary recommendations, strategically aimed at assisting you in mitigating the recurring costs attributed to your Kubernetes cluster. These recommendations aim to guide you toward optimizing resource consumption, enhancing operational efficiency, and ultimately curtailing expenses.
Pods Exceeding Resource Allocations
This subsection furnishes an exhaustive breakdown of your resources and offers tailored recommendations for Production clusters. However, you have the flexibility to opt for a distinct cluster type to acquire recommendations that are better aligned with your specific context.
- Development: Within development clusters, the objective is to maintain resource utilization at around 80%. If utilizing a 1-day timeframe, utilization is computed based on the 85th-percentile usage. Alternatively, if a different window is chosen, the calculation derives from the peak usage;
- Production: For production clusters, the target is a resource utilization of approximately 65%. If a 1-day window is selected, utilization is computed using the 98th-percentile usage. In cases of different timeframes, calculations are based on peak usage;
- High-Availability: High-availability clusters aim for a resource utilization of roughly 50%. If the analysis covers a 1-day period, utilization is derived from the 99.9th-percentile usage. For other timeframes, calculations are based on peak usage statistics.
Efficiently Manage Unused Resources
This feature tackles orphaned resources like Load Balancers, Persistent Volume Claims (PVCs), IPs, and more.
Identify Abandoned Workloads
This section sheds light on pods that have displayed minimal traffic activity over a specific timeframe, possibly indicating abandoned workloads. Once abandonment is confirmed, potential solutions encompass:
- Scaling down replicas;
- Deletion;
- Resizing;
- Notifying the pod’s owner.
| Feature | Description |
|---|---|
| Optimize Reserved Instance Utilization | Evaluates the need to purchase reserved instances for your nodes. |
| Local Disks with Low Utilization Found | Identifies PVCs with disk usage below 20%. |
| Cluster Nodes can be Right-sized | Facilitates resizing your cluster to match recent load patterns. |
| Manage Unclaimed Volumes | Presents unclaimed PVCs not associated with any pods. |
| Manage Underutilized Nodes | Showcases nodes that could potentially be removed from your cluster. |
| Health | Assesses cluster health, assigning ratings between 1 and 100. Provides recommendations and explanations. |
| Notifications | Enables Slack or Email notifications setup. Slack notifications are configured using a Slack Webhook. |
| Additional Insights | Offers a kubectl plugin for CLI users seeking alternatives to Kubecost’s WebUI. |
Conclusion
Kubecost proves to be an essential asset in the management of Kubernetes clusters. It furnishes us with a swift and effective avenue to gain intricate cost insights into our clients’ clusters, allowing us to consistently maintain budget-friendly operations while upholding an optimized cluster environment. What sets Kubecost apart is its distinctive attribute of simplicity – both in deploying the tool and navigating its functionality.