Armador: A tool for building ephemeral dev environments in Kubernetes
If you’ve been following our other blog posts at tech.travelaudience.com, you’ll know that we’re focused on running all our apps in Kubernetes. You’ll also notice that we’re big into Helm for packaging our k8s manifests. This post will get into the benefits using Kubernetes for ephemeral environments and how Armador makes use of Helm to create them.
Before Armador
When we started running our apps in Kubernetes we used an “umbrella” chart, which listed each of the microservices as dependencies in one Helm chart. The “umbrella” chart worked because it allowed for using just a single command to install all the services into an environment. But as more apps got released into k8s and they demanded their own release cycle, the umbrella chart was no longer scalable. So we broke it apart, and each app was managed with it’s own CD pipeline.
Developers now had an easy way to deploy their app into staging/production, but what we didn’t have was somewhere to test the full system. A key aspect of a microservice architecture is to make sure the individual services work in isolation, but it’s also important to make sure the service works in the full system. Providing developers a way to run a multi-service environment on their own machine proved to be complicated.
We started by looking at the some of tools in the Helm ecosystem that manage environments and dependencies, like Helmfile, Helmsman, and Terraform Helm Provider. We wanted something that would allow:
- the same (or similar) configuration to be used in different namespaces,
- the configuration of which charts to install to be easily adjustable,
- default the configuration to match what’s in production, but allow for that to be changed,
- referencing additional values files (that already exist) for the charts in the configuration.
While these tools provide some great features, they didn’t really cover all our requirements. So we began work on Armador…
Armador
The idea behind Armador is to provide something comparable to Docker Compose, but for Kubernetes. Armador installs a set of Helm charts into a particular namespace. This allows the option for multiple “environments” to be installed into the same cluster, isolated by the different namespaces.
Docker Compose provides an Example Voting App designed to provide an understanding of the way a distributed application can be pieced together. And so, Armador comes with an example directory that extends the Example Voting App. The original voting app is relatively simple and extremely over-architected, but that helps to make it a good example of what can be done with Armador. Here is a diagram of how Armador manages to deploy the Example Voting App.
Just like the original voting app is over architected, so too is this diagram. In a real world example, the dependency tree wouldn’t be as linear as the voting app. And so in creating this Armador example, the term “dependency” doesn’t map directly to what depends on what, but rather shows what can be done to build a more complex structure.
You’ll notice that we added a new file to the vote app Helm chart called armador.yaml. This file describes the dependencies required for our app. When Armador creates a new environment for the vote app, it will parse this file and collect the dependencies (redis and result )to be installed. And since result also has an armador.yaml file, the dependency tree fills out. Once the process is done, you have a full install of the app and dependencies in their own namespace and isolated from anything else that runs in the cluster (ie, any another vote app).
Needing the armador.yaml file in the Helm chart means that Armador works best with internal/private charts. But if you dig into the source of the example, you’ll notice that it’s only the vote & result charts that are defined there, while the other three charts are pulled from external sources and not modified by the extra file.
Benefits
One of the big benefits that we have found is that now the developers can make use of other Kubernetes tooling to debug their apps. For example, in the Armador repo there’s instructions on how to use Telepresence, as well as (my personal favorite) using Ksync & Squash to debug applications. Being able to trigger breakpoints in your IDE from an app that you’re working on (including the subset of service installed by Armador) is a game changing experience.
Plug: I’ll be giving a talk at CloudNative London 2019 on this topic of debugging apps in Kubernetes
Plug 2: There’ll be a lightning talk on Armador at Helm Summit in Amsterdam
Future
Armador is now open sourced under the MIT license.
While consideration was made to make it usable for any organization, it’s very much focused on how we work at travel audience. That being said, feedback, issues, and PRs are very welcome.
It should also be stated that since we started on Armador, there’s been two other tools to come out that work differently, but produce a similar result of an ephemeral environment:
* Garden
* Acyl
