Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blog post on the kubernetes future and linux kernel way #49163

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions content/en/blog/_posts/inevitable-future-of-kubernetes
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
layout: blog
title: "The Inevitable Future of Kubernetes: Why the Orchestrator Should Follow the Path of the Linux Kernel"
date:
slug: inevitable-future-of-kubernetes
author: >
Timur Tukaev (Cozystack by Ænix)
---

At KubeCon + CloudNativeCon in Chicago on November 9, Tim Hockin, one of the early developers of Kubernetes, delivered [a talk](https://www.youtube.com/watch?v=WqeShpaztZY) ([here’s a summary](https://thenewstack.io/tim-hockin-kubernetes-needs-a-complexity-budget/)) highlighting one of the orchestrator’s major challenges: the relentless growth in complexity. His main point was straightforward: Kubernetes is being applied to an ever-widening range of specialized use cases, such as machine learning.

As a result, user demands on K8s keep increasing, developers strive to keep up, and Kubernetes becomes so complex that it creates two significant problems:
1. Kubernetes developers themselves struggle to navigate the growing feature set and functionality of the project. The codebase becomes overly intricate even for those building it.
2. Engineers implementing and maintaining Kubernetes in organizations find it increasingly difficult to master all its options and configurations.

To address this, Tim proposed introducing a "complexity budget." This approach would allocate a fixed "budget" for the project's complexity, where each new feature in a release would consume part of that budget. In his view, this could help control the project’s complexity.

While this idea is reasonable, I believe the broader ecosystem—including Kubernetes developers, businesses leveraging it, and engineers deploying it—needs to rethink their expectations of an orchestrator in the coming years.

Today, Kubernetes is often seen as a standalone and (loosely speaking) self-sufficient piece of software. Yes, to use it in production, you’ll need to integrate various cloud-native tools like CNI, service meshes, and others. But Kubernetes is still commonly perceived as an application—some even call it the "OS for the cloud."

In my view, this understanding of Kubernetes is leading the industry into a dead end. It’s clear that the complexity of the orchestrator will continue to grow, and it’s equally clear that Kubernetes will be applied to an increasing number of domains, all of which stand to benefit significantly from its adoption. To remain a successful product and maintain its leadership, Kubernetes must adapt to meet the demands of these domains.

The market needs to start viewing Kubernetes as akin to the Linux Kernel. What do I mean by this? A sysadmin at a small or medium-sized company would hardly consider bypassing a Linux distribution to assemble their own from scratch using the Linux kernel—it’s simply not the norm. Everyone understands the value of choosing a ready-made distribution tailored to their specific needs.

There are general-purpose distributions that work reasonably well across different use cases, but there’s also a vast array of specialized distributions like Talos Linux, Kali Linux, VyOS, DSL, or SLAX, each optimized for particular tasks.

Moreover, distributions come with convenient package managers, making it easy to add the software you need.

Kubernetes needs a similar paradigm shift: instead of being seen as a singular product, it should evolve into the foundation for a diverse ecosystem of tailored distributions, each built to serve specific workloads and industries efficiently.

Broadly speaking, a Linux distribution is made up of several key components:
- A package manager.
- A kernel bundled with GNU utilities.
- Dedicated repositories with vetted and tested software.
- Defined release cycles.
- Kernel optimizations tailored for specific use cases.
- A mix of command-line and graphical interfaces, configuration managers, and other tools.
- A community and/or development team that ensures the system is continuously updated and maintained.

In my view, this model is something the CNCF and the broader market should adopt. To fully meet the diverse needs of modern technological domains, Kubernetes must evolve. It should stop being perceived as a standalone piece of software and transition into the equivalent of the Linux Kernel: a complex foundation known and directly managed by only a limited number of specialists.

In this paradigm, platforms—both open-source and proprietary—will serve as the "Linux distributions" of Kubernetes. These platforms would deliver Kubernetes bundled with an array of extensions, including both cloud-native tools that enhance its core functionality and additional services like databases or Kafka. This would allow the platform to be used directly for building environments tailored to specific tasks.

For example, if you want to run ML/AI workloads, you could choose from a dozen "distributions" pre-configured for this purpose. These would come with Kubernetes and related components optimized with default settings and user-configurable options to fit your needs. Alternatively, you could opt for a general-purpose distribution that can adequately handle a broader range of workloads.

Under such a model, SREs and other technical specialists wouldn’t need to dive deep into Kubernetes internals or handle full-stack infrastructure configuration. Instead, they could focus on solving business-critical, product-oriented challenges—leaving the foundational infrastructure concerns to the platform.

Today, we’re far from this vision. While numerous cloud platforms exist, they’re often perceived as alternatives to vanilla Kubernetes. They’re compared not only against each other but also against Kubernetes itself or against infrastructure built manually from "source components." Engineers frequently reject pre-built platforms, arguing that they’re poorly designed and claiming they can build something better from scratch.

We also often hear concerns like, “I know how everything works here, but over there, it’s a black box—or worse, an open box where the code is indecipherable.” This mindset reflects the current struggle to move beyond Kubernetes as a standalone tool and embrace it as the kernel of a new ecosystem.

A shift toward a "Linux Kernel for Kubernetes" approach could unlock greater efficiency, scalability, and specialization, making Kubernetes platforms more accessible and impactful for a wide range of industries.

Though many engineers recognize they could configure everything manually, they still choose opinionated distributions because these solutions efficiently handle a defined scope of routine tasks. This frees engineers to focus on solving business-critical challenges rather than infrastructure minutiae. Business leaders and owners also appreciate these preconfigured distributions for their ability to streamline operations and drive value.
However, I believe this view of Kubernetes limits both the orchestrator’s evolution and the growth of the cloud-native ecosystem as a whole. The reliance on engineers and their skillsets becomes the bottleneck for building and maintaining infrastructure.

How many engineers fully understand the intricacies of the Linux Kernel or the specific optimizations applied to a particular Linux distribution? The reason the industry doesn’t expect such knowledge is that the market has accepted Linux distributions as standard units of business value. Breaking this abstraction down into the kernel or its individual components serves no practical purpose.

Of course, large enterprises sometimes create their own distributions—Microsoft, for instance, developed Azure Linux. But for the vast majority of companies, this isn’t necessary or feasible.

Shifting the perception of Kubernetes from a standalone solution to a central component of larger platforms requires several significant steps:
1. CNCF groups like TAG App Delivery, the Technical Oversight Committee, maintainers, and major Kubernetes sponsors could initiate discussions on the future of Kubernetes. These talks should evaluate both opportunities and risks, ultimately shaping a strategic roadmap. Transitioning Kubernetes from standalone software to a component of broader platforms would require major adjustments.
2. CNCF groups should actively engage the market, advocating for a diverse ecosystem of platforms. This includes encouraging competition with existing players and creating guidelines or standards (possibly through a new working group) to simplify platform development. Kubernetes’ future as a central component hinges on enabling other developers to build platforms around it efficiently.
3. Talented engineers and companies that currently build internal platforms could shift toward developing public platforms, whether open-source or proprietary. This diversification would expand the market and foster innovation.
4. Major players in different industries could collaborate on platforms tailored to their unique requirements or create new ones optimized for their markets.
5. CNCF and other open-source organizations should adopt and nurture such projects, helping them grow and allocating time to promote this approach at flagship events like KubeCon.

The result would be a robust ecosystem of ready-made platforms that comprehensively address the needs of companies building public clouds (service and hosting providers) as well as private clouds, bare-metal deployments, and air-gapped environments (end-user companies).

At Ænix, this is exactly what we are working toward. That’s why we are striving to transfer our Cozystack platform to CNCF. We see this as a signal to the market that this transformative process has already begun. In our view, this step will catalyze changes that benefit all market participants.

I welcome feedback, clarifications, suggestions, or any thoughts you might have ([my LinkedIn](https://www.linkedin.com/in/tym83)). This isn’t a fully developed, market-tested concept—it’s the beginning of a public dialogue about a critical problem. Let’s discuss and refine these ideas together.