DelightRoom DevOps Part 1: What the Heck is Argo CD?

What is Argo CD?

Introduction

Hello. I'm Chris, an engineer in the Infra Group handling backend development at DelightRoom. Our group is constantly exploring various DevOps technologies to operate and deploy our backend services more systematically and efficiently. One of our core components is the deployment management tool Argo CD, which we adopted and have been using for a few years now. I decided to write this post because I want more people to know about its benefits and give it a try. I'll cover the background of why we adopted Argo CD, its main features, and some use cases here at DelightRoom.

This post assumes you have some basic knowledge and experience with Kubernetes. However, I won't be diving too deep into the technical weeds, so even if you aren't super familiar with it, you should still be able to walk away with a decent understanding of Argo CD and GitOps.

What is Argo CD?

Argo CD is a tool that supports application deployment and management in Kubernetes environments, built on GitOps principles. At its core, it synchronizes the application state recorded in a Git repository with your Kubernetes cluster. This allows you to effectively track and manage the entire deployment state and any changes right through Git.

Background of Adoption

In DelightRoom's tech stack, Kubernetes is a crucial foundational environment where the Alarmy service API, internal services, and data pipelines all run. Stable deployment of these components is a core business requirement, which led us to the following considerations:

1. The state recorded in our Git repository must match the current state of our Kubernetes cluster.

2. The typical deployment flow involves building an image, uploading it to a container registry, defining it as a k8s resource, and applying it. However, this method is cumbersome because it requires reapplying resources every time there's a change.

3. Verifying state consistency requires manual effort, which is not only tedious but also leaves a lot of room for human error. Therefore, we decided we needed to adopt an automated synchronization approach through GitOps.

With this background in mind, we explored several related tools (such as Flux, Jenkins X, Argo CD, and Tekton), and ultimately decided that Argo CD was the best fit for us.

Why We Chose Argo CD

Our main reasons for choosing Argo CD were as follows:

1. Efficient synchronization of state between Git and Kubernetes.

2. A clean and intuitive installation and operation experience.

3. Seamless integration with various tools (like helm, kustomize, etc.).

4. The convenience of declarative management.

5. Active community support and rapid feature updates.

Thanks to these advantages, Argo CD was selected as the most suitable deployment tool for DelightRoom's Kubernetes environment.

It wasn't quite at this level when we first started using it, but today Argo CD is used so extensively that it's widely considered the de-facto standard among open-source GitOps deployment tools.

Installation and Operation Approach

When introducing a new technology, complex configurations can often become a barrier to entry. In our early days of adopting Argo CD at DelightRoom, we started by quickly installing and testing it following the clear official installation guide.

Transitioning to Kustomize and Declarative Operations

As time passed and we introduced IaC, we wanted to transition our Argo CD management approach to something more systematic and declarative. To do this, we utilized `kustomize` to specify the entire state of Argo CD (including various settings, Application or ApplicationSet definitions, etc.), stored it in our Git repository, and set it up so we could bootstrap it all at once. During this process, we heavily referenced the following Declarative Setup guide.

Recursive Operation of Argo CD

The flexibility of Argo CD even allows for recursive operation, meaning you can manage Argo CD itself using Argo CD. This approach has the advantage of making configuration and management even more systematic. There are plenty of use cases for this in various organizations, but at DelightRoom, we are currently still considering taking it to this level.

Use Cases

1. General Backend Service Deployment

In modern DevOps environments, it's standard practice to deploy software quickly and safely across multiple stages (dev, stg, prod, etc.). Deployments must happen while changing various configurations—like image tags, resource constraints, scaling options, and environment variables—depending on the environment. Argo CD makes it quite easy to support this.

The most basic approach is to create a separate Argo CD Application for each environment. You can think of an Application as the fundamental unit of deployment in Argo CD. A single Application holds various configuration details, such as the location of the resources to be deployed in the cluster, the deployment method, and the target namespace or cluster.

A more advanced approach involves utilizing Argo CD's ApplicationSet. An ApplicationSet is essentially a tool that allows you to template Applications and generate them automatically. It minimizes the repetitive work required when deploying a single Application differently across various environments. If you're curious for more details, check out the official documentation.

We'll cover exactly how we actually use Application and ApplicationSet, along with technical details, in a separate post.

2. Deploying Foundational Services to All Clusters

Certain foundational common services (such as ingress-controller, agents for observability, autoscaler, etc.) need to be deployed across the board regardless of the cluster type.

The ApplicationSet mentioned above is a highly useful tool for solving this deployment challenge. Beyond simple batch deployments, we often need to apply different configuration values depending on the environment. By utilizing additional helm values files or ConfigMaps, you can deploy the same service while applying environment-specific settings.

By leveraging Argo CD and ApplicationSet this way, it becomes possible to effectively and consistently deploy necessary services across various clusters. Again, we will cover the more detailed technical aspects in a follow-up post.

Pros and Cons We Found Using Argo CD

Pros

One of the biggest advantages of GitOps practice via Argo CD is that "Code = Cluster State." This allows us to clearly understand the current state of the cluster and manage it without communication hiccups. For instance, if one engineer locally applies a specific manifest via kubectl, and another engineer applies a different manifest, confusion can easily arise over which one was actually applied. With multiple engineers, tracking these changes becomes nearly impossible. However, using Argo CD makes it easy to overcome these issues.

In addition, Argo CD manages applications comprising multiple Kubernetes resources as a single 'Application' unit, and even provides a 'project' concept on top of that, helping to systematically manage the deployment state of various services.

Lastly, as of 2023, Argo CD is a very active open-source project recognized by numerous users, including global tech giants like IBM and Intuit (see here: https://github.com/argoproj/argo-cd/blob/master/USERS.md). On the security front, it offers various configuration options, allowing you to build robust security frameworks as needed.

Cons

We've also noticed a few downsides while applying it in our production environment.

The most noticeable issue is that it's difficult to make the entire state of the cluster fully declarative. For example, if there are dependencies between applications, you can't specify deployment order within Argo CD, which means you have to manually solve any resulting issues yourself. There's also a high chance of running into the same problem when importing and using external tech stacks like databases or Ingress Controllers. Depending on the technology, they often have their own specific deployment methods, and these don't always translate cleanly into a declarative format. These issues sometimes resolve themselves over time, but until they do, you might have to include imperative elements in your deployment process.

Furthermore, Argo CD isn't a native Kubernetes component, nor is it a commercial solution—it's a third-party open-source project. This means you have to take responsibility for everything, right from installation to operation. If a problem occurs, it's hard to get official support, so you have to rely on the community. The UI and usability might also be less intuitive compared to commercial solutions. Thankfully, there are commercial GitOps solutions based on Argo CD available on the market, so you can choose to use those if needed.

Conclusion

I've introduced Argo CD and GitOps, which form the backbone of DelightRoom's backend DevOps. Argo CD has made the deployment process in our Kubernetes environment far more efficient. Through automated synchronization, we've been able to pursue both stability and efficiency at the same time. I hope the contents of this post will be at least a little helpful to anyone out there looking to build backend infrastructure in the future. In my next post, I'll introduce specific application use cases alongside actual code.