Why Kubernetes for Serverless?

Many people would love to build a serverless platform in their own tenant. After all, the whole premise of platform engineering is to enable developers to self-serve on your organization’s platform, be it in the cloud or on-premise. Serverless gives you that flexibility, but it can be hard to figure out how to build this platform yourself.

What if there were an existing pattern or toolkit that could make this happen?

My Thoughts and Experiences with Serverless

I love serverless. This should be obvious from my other posts but I also love Kubernetes. I remember first learning about Kubernetes at a Meetup in Austin back in 2015 if not mistaken. I was just learning about containers and I was also trying to make a career pivot and enter the DevOps space. While the company I was working for at the time wasn’t using containers, I experimented with them on my own time and I found them very useful and quickly became a fan.

That being said, I have also always loved serverless. I remember first learning about Lambda at an event at SxSW in 2015. This was the first time I personally heard the term “serverless” but the concept intrigued me. The ability to focus on code instead of servers.

I am old enough to remember buying a computer at a garage sale and making it an Apache server. I hosted my website as well as the websites of a few friends. But obviously, this was a lot of work so I eventually moved to a VPS provider. I still wanted command line access to the machine but I didn’t want to deal with having hardware in my room and maintaining the network. I also didn’t want to deal with the hardware failure and the associated downtime.

When I heard about serverless, the ability to focus on code instead of servers I was stoked. I was already and early adopter of Google App Engine but you still had to worry about workers and stuff like that. Lambda was like “hey just upload your Javascript (at the time) code and we’ll make sure it runs. Oh ya, you only pay for when the code is running”.

I have had people ask how I reconcile these two loves or why I would even want to. Like one concept requires infrastructure. It abstracts infrastucture management for operators but it really isn’t developer centric like the other concept. Well I am here to tell you that it isn’t really and “either-or” kind of deal.

Kubernetes is the perfect platform for running serverless workloads and here is why:

Serverless Has Servers

Serverless is a misnomer. I often say that we haven’t figured out a way to host applications on actual clouds yet. Of course there are servers and routers and hard drives in a serverless platform. The keyword there is platform. Serverless is a platform that abstracts away all of the infrastructure management for developers. At the end of the day, a developer wants to build and deploy code.

This requires some kind of middleware that can automatically deploy code onto a server. When I had my server in my room, I wrote a small script that would check for code as I sent it via SFTP. The script would then move the code to the right folder and update the Apache configuration. This was fine for a small HTML website managed by one person but it would not be great for a large application with multiple developers.

There are many ways one could achieve this. In the age of cloud, you also need a way to scale and distribute this middleware and the code. Chances are high that you are not running your enterprise application on one huge server. You are probably running it on multiple servers and likely in multiple regions.

Enter the World of Kubernetes

Kubernetes is amazing because it allow you to run workloads across multiple servers. Now is Kubernetes the only tool for orchestrating workloads across a distributed system? Of course not. I don’t even think it’s fair to call it the first. However, it is undeniably the most popular.

In fact, per a Cloud Native Computing Foundation survey, Kubernetes is used by 83% of organizations. Now this post isn’t about how amazing Kubernetes is. It is about how useful Kubernetes can be for serverless platforms.

People often state that Kubernetes is too complicated or “low-level” for developers and I tend to agree. It does provide a lot of infrastructure abstraction and simplifies the management but it still requires some level of infrastructure management. The below word cloud is a simple summary of the things that a developer would have to learn to leverage Kubernetes barebones.

Obviously making a developer learn even half of these concepts just adds “busy work” to their job. It’s important work, don’t get me wrong, but busy work. So again, why am I talking about Kubernetes and developer-centric platforms?

Well one of my favorite quotes comes from Kelsey Hightower where he states that “Kubernetes is a platform for building platforms”.

Using this mindset, Kubernetes is a platform for building serverless platforms. One thing that makes Kubernetes a great platform is the Cloud Native Ecosystem. You may have seen this landscape map from the CNCF:

Apologies for the ee chart there. As time progresses, that map just gets bigger and bigger. This map shows all of the different projects that are part of the CNCF and it doesn’t even include all of the projects available for Kubernetes. These tools can provide platform engineers with the tools that they need to build a serverless platform for their developers.

How Do I Build Serverless Platforms on Kubernetes?

So lets bring this all together. How would I build a serverless platform on Kubernetes? Well first we need to remind ourselves of why we are building a serverless platform in the first place.

We want to empower our developers to focus on writing code and being able to deploy it without having to worry about the underlying infrastructure or file endless tickets to an operations team.

With the vast ecosystem that Kubernetes provides, it is entirely possible to automate the deployment of code onto the cluster. With CNCF projects such as Buildpacks, you can even remove the Dockerfile from the equation. Developers can create containers without even knowing what a container is.

When it comes to deployment and scaling, you can use projects such as Knative or KEDA to “automagically” handle the scaling and routing of your application. Backstage can be used to provide a developer portal for your developers to interact with your serverless platform. Dapr can be used to provide a set of APIs for your developers to interact with different databases and message queues and third-party tools which simplifies management.

I am barely scratching the surface as there are a multitude of ways to build a serverless platform on Kubernetes with the various tools available. You could even leverage something like Pulumi to turn infrascture into programming languages like Typescript or Python. You can even create your own Kubernetes Operators to extend Kubernetes.

With the various software platforms out there, a developer can leverage the benefits of containers, microservices, and the scalability of Kubernetes without even learning anything about Kubernetes. In an ideal world, serverless completely abstracts away the infrastructure in such a way that developers have no idea what’s going on under the hood.

Conlcusions and What’s Next

Ultimately the goal here was to show you how platform engineers and Kubernetes operators can leverage Kubernetes and the CNCF ecosystem to build serverless platforms in their own environments. Many people feel like they can only leverage serverless platforms by working with a third-party cloud provider.

While there are definitely benefits to using a provider, sometimes you want to host stuff in your own tenant. Perhaps you are hybrid or multicloud. Maybe you are in your own datacenter and have multiple legacy components. There are multiple reasons why you would want to host your own serverless platform and Kubernetes makes it possible.

In a future post, I will dig deeper into what I believe is an ideal toolkit and architecture for building a serverless platform on Kubernetes. Until then, what are your thoughts? Do you use Kubernetes for serverless platforms? What tools do you use? Let me know and I will see you online.