Revisiting Serverless Eventing

Back in 2020, during those COVID years, I did a blog series on what I was calling “Serverless Eventing”. I didn’t really invent the term, but I found it to be a fun topic to talk about. Early in my career at Google, I got to play with Apache Kafka quite a bit. Then Google created and open-sourced Knative Eventing.

I decided to play with this in detail and this led to my blog series. Then life caught up with me and I kind of just stopped. For the past 6 months or so, I have toyed with the idea of bringing back the series and I finally decided to do so.

Here’s why I think Serverless Eventing is more important now than ever.

Serverless is Growing

According to a report from Persistence Market Research the [serverless] market is expected to grow at a CAGR of 15.7%, reaching approximately US$ 58.95 billion by 2031. This is significant growth since I stopped writing about this topic circa 2021.

When I was writing, serverless was largely just functions with a few container based solutions like Knative, Cloud Run and Fargate but that was it. Nowadays we have serverless databases such as Neon which is reportedly in talks of aquistion by DataBricks I might add as well as serverless inferencing companies and more.

WebAssembly(WASM) is positioned to bring a new serverless paradigm in terms of cloud-native packaging. Serverless has evolved far beyond simple functions and has evolved beyond compute. With more serverless, it only suggests more eventing.

By the way, if you want to keep up with serverless news, checkout my newsletter, The Cloud Is Serverless.

Event-Driven Architecture is Becoming more Important

In my earlier posts, I talked about how important event-driven architecture was. Events run our technical world. Make a purchase online, a message gets sent to CRM which triggers an email to the customer and an update to inventory which sends another trigger to the warehouse, and so on. So many events take place within enterprise applications.

I think that we are now nearly 4 years in the future, more and more applications have become cloud-native and have adopted a microservices approach. But we are also seeing applications evolve to need more and more message streaming services and therefore more eventing.

One of my favorite channels to follow is ByteByteGo and they have a great post talking about different messaging. You can see that there are many . We also have newer technologies being used such as Apache Pulsar, Apache Flink and Apache Storm. Seems like the Apache Foundation really loves streaming.

Then there are also startups like Redpanda out there that specialize in streaming data. Clearly there is a need for more message streaming solutions. Event-driven architecture is needed to help real-time applications simply work.

More Eventing Platforms

While message streaming services are growing and serverless is growing, we need more platforms to handle eventing. In the proprietary world, since I last wrote, many tools have come about such as AWS Event Bridge, Azure Event Grid and Google Cloud EventArc.

In the OSS world we have Knative Eventing of course but others such as TriggerMesh and KEDA to some extent. I have seen a few others comes about but these are the “more mature”. Even still, we see a lot of this development taking place.

Agentic Apps and Workflows are in Fashion

Okay, so I can’t avoid the elephant in the room. Ever since late 2022, every other tech conversation has been about AI. It makes sense, OpenAI’s release of ChatGPT opened the floodgates for Generative AI and its applications. People are making applications that inference with LLMs and do all sorts of interesting things.

Sounds a lot like event-driven architectures and a great case for serverless eventing. I mean, you have these applications that are trigger by a user or another service that then have to send a message to inference with an LLM and you likely have an inferencing gateway there. Then it gives a response and often, next steps are to do something else triggering another event.

I also fully believe that serverless is the best architecture for inferencing with models (inferencing, not training… yet). The application can spin up when it is called, inference with the LLM, and return a result. What if you can declaratively bind all of this?

Agentic AI is currently doing something similar. Late last year, Anthropic released Model Context Protocol which acts as something of an intermediary between your AI Agents and the various LLMs. This is very similar to how Knative Eventing works as an intermediary between your application and the various event sources.

Serverless Eventing should be able to slip into this new MCP ecosystem as they are effectively the same thing.

It Never Stopped Being Important

Ultimately, you can tell from my post that serverless eventing never stopped being important, and with the growth of serverless and event-driven architecture mashing up with the growth of Generative AI, Agentic Apps, and MCP, we are seeing Serverless Eventing being more important.

So What’s Next?

I plan on writing a blog post at least once a month (I will aim for twice a month though) covering both AI and non-AI topics as it relates to serverless eventing. I will showcase various platforms and architectures. I will provide GitHub code and also update my old code as I haven’t done so in years.

I also plan on showing a lot of use cases and many will have Pulumi components so that you can try it yourself!

Cover Image Credit to Wendy Wei on Pexels