Kubernetes costs keep rising. Can AI bring relief?

Containerized applications offer enterprises a range of benefits in the cloud, but a dirty little secret of the industry has been that the cost of managing them hasn’t been one.

According to a recent survey, a huge majority of organizations using Kubernetes for container orchestration have spent more in the past year, with many now turning to AI to help manage costs.

About 88% of practitioners surveyed by Kubernetes management vendor Spectro Cloud say their total cost of ownership has grown in the past year, while 92% say they are investing in AI-based cost optimization tools.

These cost hikes often come with rising use of Kubernetes, also known as K8s, although increasing expenses go beyond the costs of hardware, cloud services, and support contracts, says Jeremy Oakey, field CTO at Spectro Cloud.

Employee costs also are a factor, with platform engineering salaries often approaching $200,000 a year, he adds, and other costs can include field engineering expenses for deploying and maintaining edge clusters and productivity costs when developers spend too much time managing clusters.

While increasing use is a big driver of overall cost increases, other issues, such as overprovisioning, can also add to the problem, Oakey says.

“Forecasting Kubernetes resource requirements is challenging, and developers often set generous configuration parameters just in case,” he adds. “At scale, this behavior leads to significant cluster sprawl.”

Architectural decisions create another major cost driver, according to Oakey. Data ingress and egress between clouds can be expensive, and running multi-node clusters at the edge can drive up spending.

Kubernetes teams need to be disciplined about how they use the tool, he says. They should consider whether to decommission unused clusters, downsize resources, set appropriate autoscaling thresholds, and review policies such as logging retention and storage configuration, Oakey recommends.

Poor deployment choices

Yashin Manraj, CEO of digital transformation and cloud migration firm Pvotal Technologies, sees poor configuration, overprovisioning, and unoptimized resources as the main issues for organizations spending more on Kubernetes-based environments.

“People tend to treat K8s as static VMs or servers, and this approach leaves a lot of orphaned assets and a lack of understanding as to what it actually should do,” Manraj says. “I think the primary culprit is the pressure to use K8s without having a team that understands the intricacies of orchestration or a control plane to manage clusters effectively.”

Overprovisioning K8s is the default deployment, and most teams provision CPU and memory with a healthy buffer to avoid being paged in the middle of the night, adds Yasmin Rajabi, chief operating officer at cloud management platform vendor CloudBolt Software.

“The incentives for devs and platform engineers aren’t aligned,” she says. “Devs are measured on how quickly they can deploy new apps and move the business forward. They aren’t pushed to be cost-efficient, so the easiest and safest thing to do is to make sure there’s a healthy resource buffer for workloads.”

But those buffer costs can quickly add up across thousands of Kubernetes workloads, Rajabi adds.

In addition, dynamic workloads are hard to predict, and most organizations don’t have the tools they need to continuously adjust resource requests and limits based on changing workload needs, Rajabi says. At the same time, rightsizing is a manual and time-consuming process.

“Deploying to Kubernetes is easy; managing it is not,” she says. “Kubernetes allows businesses to deploy business-impacting applications at the speed they need it with the flexibility they require to innovate and get to market faster. However, the promised ease and speed is at odds with the complexity of managing Kubernetes at large scale and the cost that comes along with it.”

Underused CPUs

Digital marketing agency NEWMEDIA.com has seen its K8s expenses go up by 18% in the past year, says Steve Morris, founder and CEO. The main drivers have been labor costs and what Morris calls scattered scaling choices. When the company reviewed its K8s autoscaling choices, it found that 31% of workloads were running at less than 25% CPU usage for 95% of the day.

“When release teams manage their own clusters and define their own [autoscaler] rules, it leads to conservative overprovisioning getting locked into their configuration files,” he says. “Engineers were reluctant to adjust resources down because they were afraid it might cause incidents or risk missing SLAs.”

Labor costs can add up when two platform engineers are spending half of their weeks tuning capacity and handling noisy alerts, Morris says. “The operations overhead grows with every new service, not just as traffic increases,” he adds. “When you let teams make their own resource decisions, it speeds up delivery, but it also splits up cost responsibility and hides waste.”

The popularity of K8s adds to the cost concerns. With Kubernetes at the heart of many customers’ IT infrastructures, rising costs aren’t likely to go away, Pvotal’s Manraj says. “We believe this problem to only rise as adoption increases with the sprawl of poorly designed tools and reliance on third parties,” he says.

Pvotal hasn’t yet found an automated solution to manage K8s costs. “We regularly test all new tools and technologies,” he says. “We unfortunately have not found any tool that delivers value to us or our clients aside from a cosmetic dashboard that tracks usage better than GCP’s default tools.”

Turning to AI

Nevertheless, the complexity of managing Kubernetes has prompted practitioners to consider AI-driven management tools, others say.

“Resource allocation is complicated and multidimensional, which really means it’s a complex math problem,” CloudBolt’s Rajabi says. “This is where AI and ML tools can help solve the complex math piece of what resources should be set to and combine with automation to actually configure the workloads with the right settings.”

A handful of vendors now offer autonomous rightsizing and intelligent autoscaling, Spectro Cloud’s Oakey says.

“We are seeing the AI cost optimization tool landscape evolve rapidly, with vendors converging from both the Kubernetes management and FinOps domains,” Oakey says. “On the management side, these tools continuously monitor real-time pod utilization, learn from historical usage patterns, and automatically adjust resource requests, node sizing, and even the balance between spot and on-demand instances.”

FinOps vendors, meanwhile, are now integrating AI and ML capabilities to enable proactive cost control measures.

“While not all of these capabilities represent AI in its most advanced form, we are seeing a clear shift toward embedding greater intelligence and automation across the entire toolchain,” Oakey says. “This convergence is creating a more sophisticated, proactive approach to Kubernetes cost optimization — one that blends operational control with financial accountability.”

CIOs should use every strategy available to contain costs, including negotiating with cloud vendors and using autoscaling capabilities, Oakey adds. CIOs also should align broad architectural decisions with the organization’s business objectives, he recommends.

These decisions can include whether the CIO is replacing data center leases with cloud-based operational expenses or strategically reducing IT headcount by using automation to simplify operations.

CIOs, however, should also be sure to distinguish between costs and investments, particularly in the AI era, he adds.

“Deploying Kubernetes clusters to support transformative, revenue-generating applications is not just an expense; it is an investment in innovation and competitive advantage,” he says. “The central question then becomes whether your Kubernetes infrastructure is being operated to deliver maximum value in areas such as speed, choice, and flexibility for your application teams.”