This is a saga about how Livspace revolutionised it's software release orchestration using one of the best tools in the business — Devtron.

Let us tell you about the saga of how Livspace evolved as a tech-first company to revolutionize our software release orchestration. How we started with the good ol’ VMs and moved on gradually to using one of the better tools in the business — Devtron.

As a tech-first company building a three-way platform, our tech stacks are rather flavorful: python, java, nodejs, golang, among others.

Our core is a platform; it is not a single toolkit, but a collection of toolkits on a singular platform. Think of us as an operating system, but for interior design and home renovation.

Having a robust DevOps process and appropriate tools for it, that caters to this variety of use cases, thus becomes a “p0 bug” that we need to resolve.

A brief history of CI/CD

In software engineering, CI/CD or CICD is the combined practices of continuous integration (CI) and either continuous delivery or continuous deployment (CD).

Let’s start with a very brief introduction to what CI/CD is. CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in the building, testing, and deployment of applications. The process contrasts with traditional methods where all updates were integrated into one large batch before rolling out the newer version. Modern-day DevOps practices involve continuous development, continuous testing, continuous integration, continuous deployment, and continuous monitoring of software applications throughout their development life cycle. The CI/CD practice, or CI/CD pipeline, forms the backbone of modern-day DevOps operations.

The Livspace CI/CD Saga

Arc 1: A Beginner’s Journey

As most startups begin their initial journeys, we started our Tech by building a handful of monolith services and deploying them directly on virtual machines (VMs) like EC2 instances (good old days 🥲).

As we started to migrate towards a microservice architecture, the number of microservices started increasing. We soon realized that this would highly impact our scalability and fingers to production time unless we soon move to a CI/CD platform.

Arc 2: The Novice’s Adventures

When we first realized that the existing model would hamper our microservice architecture, we began the great search for orchestrating our continuously growing services. Around the same time, Kubernetes had already spread like wildfire in the tech community and as lovers of the open-source community, we implicitly moved towards leveraging it rather than opting for any paid solutions.

Kubernetes (K8s)is a tremendously powerful tool for orchestration. But as Uncle Ben said,

It needs to be noted that deploying applications to K8s is much more challenging than running a server start-up command on a VM. One of the biggest challenges is the know-how of converting your application into a container image (e.g. Docker) and deploying it as a set of yaml manifests (e.g. Deployments). Today this might look like an easy problem to solve with the plethora of CI/CD platforms available like Devtron (more on this in a bit), CircleCI, Flagger, etc, but back in the day, we had to build our own solution from scratch.

At the time we leveraged a now well-known K8s package manager, Helm, as our base for templatizing YAML manifests for our applications. Also, we relied on Jenkins as a CI tool to build and pack our apps into Docker images. Painstakingly, we wrote all the scripts necessary to achieve this custom CI/CD solution.

And this custom solution worked well for us for almost a couple of years ✌️, until we were again shown the mirror by the rapid advancement of technology in the sands of time.

Arc 3: The Setback

With the microservice architecture serving as the backdrop, we continued to grow in leaps and bounds. While we started with a handful of microservices, we quickly grew to 100+ microservices and are now looking to grow exponentially to few hundreds in the time to come (SRP FTW!). We started with a single K8s cluster, and are now already managing multiple clusters. Soon we would be going multi-region with more K8s clusters to be added to the fray.

Also during this time when we focused more on dev than DevOps, K8s had already jumped few versions. We were now looking at maintaining a CI/CD solution that was tightly coupled with a legacy K8s version that had already reached its EOL (end-of-life). This gave us a grim reminder as to where we stood, and that we were in dire need to overhaul our existing CI/CD solution so as to cater to these titanic needs of the future.

Arc 4: A Chance Encounter

Just when we were trying to figure out whether to rewrite our existing custom CI/CD platform or to opt for an open-source one (since we ❤️ open-source, not sure we mentioned that), we had a chance encounter with Prashant Ghildiyal, co-founder of Devtron Labs, who with his fantastic team was building,

an open-source release orchestration & CI/CD platform for Kubernetes.

After our first encounter with them, we immediately hit it off. Having gone through a similar curve as us, they vividly understood our pains. With that in mind, they had already built a great platform as a solution to all the problems I mentioned earlier.

What is Devtron?

In their own words,

It is designed as a self-serve platform for operationalizing and maintaining applications (AppOps) on kubernetes in a developer friendly way.

It provides a plethora of super-cool features. Out of which, the one that we think creates the biggest impact is their unified interface, which we also call The Dashboard. Tons of open-source alternatives that are out there do a great job of deploying apps to K8s these days, but very few to none have such an intuitive dashboard that encapsulates everything that an application owner needs to deploy and debug their apps smoothly.

It became an instant hit as soon as we launched it for our internal application teams. They were able to relate to it so much that it became an absolute necessity for all of us. We immediately saw the following benefits after the launch:

1. One is improved TAT in the overall CI/CD pipeline, from building to deployment.
2. This is possible via their fine-grained access control that, importantly, app owners can do, without any DevOps team involvement.
3. Business-focussed dev teams themselves can debug build, deployment, and even real-time application issues in a single place.
4. Uses gitops but it is much easier than before.
5. Has great support for integrating helm charts which have now become synonymous with hosting open-source software on K8s.
6. Spawning new namespaces and environments is a breezier task than ever before.
7. Makes future upgrades to clusters simple via their multi-cluster support.

Arc 5: The Comeback and Beyond

All these factors ushered in a new era in our Tech ecosystem. We started migrating all of our apps on to Devtron and were able to soon onboard 150+ microservices in a matter of few weeks. Earlier we couldn’t have thought of achieving it in months (maybe even quarters 😅).

The turn-around-time (TAT) that I briefly mentioned above has also reduced dramatically. Software releases to production which used to take months or sometimes quarters earlier, have now come down to nearly a week and even less than a day for some services.

We measure the TAT of a service using the following parameters:

• Deployment Frequency: How often is this app deployed to production?

• Change Failure Rate: How often does the pipeline fail?

• Mean Lead Time: How long does it take to deliver a change to production?

• Mean Time to Recovery: How long does it take to fix a failed pipeline?

Again all these metrics come bundled with Devtron (voila!), so you do not have to spend any extra time in capturing or projecting it. And these metrics are available for each service which means each service owner can easily figure out the areas where they are excelling or lacking in terms of agility. For example, we were able to quickly improve the TAT for one of our services named bumblebee using these metrics.

This also encouraged us further to start migrating our grand-old legacy K8s cluster onto a brand new EKS cluster. Surprisingly we were able to achieve it in even lesser time. We migrated multiple staging environments (yes entire environment of 150+ services) in less than a week!

via GIPHY

This has been a true revolution for us, where now we are much more agile compared to earlier, have faster release cycles, faster (issue) debugging and patch releases, more flexibility in terms of K8s features usage, and much more scalability in terms of spawning new clusters and running our apps on the go. To top it all, Devtron is open source and has a great community, and you already know by now how much we ❤️ open-source (in case we missed mentioning this).

That’s it for today folks! Here’s a cookie for the long post!🍪