Skip to main content

Argo CD GitOps with ApplicationSets

·1942 words·10 mins
Milad Zangeneh
Author
Milad Zangeneh

ArgoCD GitOps deployment overview

I recently set up a GitOps workflow for a few Python microservices I’ve been working on, and I wanted to share how I did it. The first version used a list of services inside each ApplicationSet. That worked, until every new app meant editing YAML in two places and hoping I didn’t typo a chart name.

What I landed on instead is simpler to live with: the directory tree is the inventory. An ApplicationSet watches a path in git, and each folder it finds becomes an Argo CD Application. Adding a service is adding a directory. No ApplicationSet edits.

What I Started With
#

I had a handful of Python apps, each with its own Dockerfile, images sitting in a private registry, and a Helm chart published to a chart repo. Each service in git is a thin umbrella: a Chart.yaml that depends on that remote chart, plus a values.yaml with the overrides for that environment.

The missing piece was deploying them in a repeatable, Git-driven way (including a real prod/dev split, and a separate place for cluster services like Grafana that don’t belong in the same lifecycle as the business apps).

The Idea Behind the Setup
#

Instead of listing every service in an ApplicationSet (or running helm install from CI), I wanted a repo where Argo CD discovers apps from the filesystem. Two trees, because they don’t change for the same reasons:

  • environments/: business apps. Each one ships in prod and in dev, so each has a copy under both. They share one cluster, isolated by namespace.
  • platform/: shared cluster services (monitoring, dashboards, that kind of thing). There’s only one of each, so they skip the prod/dev split.

The path encodes everything the ApplicationSet needs. For a business app that’s environments/<env>/<namespace>/<service>. For platform it’s platform/<namespace>/<service>.

platform-gitops/
└── argocd/
    ├── root.yaml
    ├── projects/
    │   ├── production.yaml
    │   ├── development.yaml
    │   └── platform.yaml
    ├── applicationsets/
    │   ├── production.yaml
    │   ├── preproduction.yaml
    │   └── platform.yaml
    ├── environments/
    │   ├── prod/
    │   │   └── application/
    │   │       ├── app-a/
    │   │       ├── app-b/
    │   │       └── app-c/
    │   └── dev/
    │       └── application/
    │           ├── app-a/
    │           ├── app-b/
    │           └── app-c/
    └── platform/
        └── monitoring/
            └── grafana/

A few layers, each with a job:

  • A single root Application that you apply once. It only syncs projects/ and applicationsets/.
  • AppProjects as guardrails (which namespaces an app is allowed to touch).
  • One ApplicationSet per environment, so the Argo CD UI gets a clean node per env.
  • Helm umbrellas under environments/ and platform/. Those are not synced by the root app; the ApplicationSets pick them up.

Step 1 (The Root Application)
#

This is the only thing you apply by hand. Once it’s in the cluster, Argo CD watches git and keeps the projects and ApplicationSets in sync.

The important bit is the include. The root app must not recurse into environments/ or platform/ (those directories are Helm charts, not Argo CD resources). If the root tried to apply them as raw manifests, you’d have a bad time.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: platform-gitops
  namespace: argocd
spec:
  project: default
  source:
    repoURL: git@github.com:your-org/platform-gitops.git
    targetRevision: main
    path: argocd
    directory:
      recurse: true
      include: "{projects,applicationsets}/*.yaml"
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      selfHeal: true
      prune: false

I set selfHeal to true so that if anyone manually changes something in the cluster, Argo CD reverts it (a bit like how an agent-based configuration management system such as Puppet keeps correcting drift when the live system no longer matches the catalog you defined). And I intentionally left prune as false on this root app. If a file gets moved or git has a hiccup, I don’t want Argo CD deleting every ApplicationSet (and with them, every live service) in one go.

Apply it once:

kubectl apply -f argocd/root.yaml

Argo CD also needs credentials for the git repo before any of this can sync. If the repo isn’t registered, the Applications get created and then sit there unable to pull.

Step 2 (AppProjects)
#

I skipped projects the first time around and dumped everything into default. That gets messy once you have prod, dev, and cluster services on the same cluster. Projects are the guardrail: a production app cannot target monitoring, and Grafana cannot land in application.

AppProjects are applied by the root app before the ApplicationSets that reference them. A sync-wave annotation on the project takes care of the order:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: production
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "-1"
spec:
  description: Production business applications
  sourceRepos:
    - git@github.com:your-org/platform-gitops.git
  destinations:
    - server: https://kubernetes.default.svc
      namespace: application
  clusterResourceWhitelist:
    - group: ""
      kind: Namespace
      name: application
  namespaceResourceWhitelist:
    - group: "*"
      kind: "*"

CreateNamespace=true on the ApplicationSets needs that Namespace whitelist. Without it, Argo CD refuses to create the destination namespace even though the project is otherwise namespaced-only.

Development looks the same, with application-dev as the destination. Platform lists monitoring (and whatever else you put under platform/), and if a chart legitimately creates cluster-scoped RBAC (Grafana does), you whitelist ClusterRole / ClusterRoleBinding there.

A new namespace directory is not enough on its own. You also add that namespace to the matching project, in both destinations and the Namespace whitelist.

Step 3 (ApplicationSets)
#

An ApplicationSet here is a git directory generator plus a template. It does not list services. It watches a glob, and each matching directory becomes an Application.

I use one ApplicationSet per environment so the Argo CD tree groups apps under a named node (production, preproduction, platform) instead of dumping everything in one pile.

Production
#

Path segments for argocd/environments/prod/application/app-a are: argocd / environments / prod / application / app-a. Namespace is segment 3. Service is the directory basename.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: production
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: git@github.com:your-org/platform-gitops.git
        revision: main
        directories:
          - path: argocd/environments/prod/*/*
  template:
    metadata:
      name: "{{ .path.basename }}-{{ index .path.segments 3 }}-prod"
    spec:
      project: production
      source:
        repoURL: git@github.com:your-org/platform-gitops.git
        targetRevision: main
        path: "{{ .path.path }}"
        helm:
          releaseName: "{{ .path.basename }}"
      destination:
        server: https://kubernetes.default.svc
        namespace: "{{ index .path.segments 3 }}"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true
          - ApplyOutOfSyncOnly=true

That turns environments/prod/application/app-a into an Application named app-a-application-prod, Helm release app-a, namespace application. The service name comes first so two apps called app-a in different namespaces never collide.

Prod Helm releases stay unsuffixed on purpose. If something was already installed under that release name, Argo CD adopts it instead of creating a parallel one.

Pre-production (dev)
#

Same idea, pointed at environments/dev. The Kubernetes namespace and the Helm release both get a -dev suffix, because prod and dev share one cluster:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: preproduction
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: git@github.com:your-org/platform-gitops.git
        revision: main
        directories:
          - path: argocd/environments/dev/*/*
  template:
    metadata:
      name: "{{ .path.basename }}-{{ index .path.segments 3 }}-dev"
    spec:
      project: development
      source:
        repoURL: git@github.com:your-org/platform-gitops.git
        targetRevision: main
        path: "{{ .path.path }}"
        helm:
          releaseName: "{{ .path.basename }}-dev"
      destination:
        server: https://kubernetes.default.svc
        namespace: "{{ index .path.segments 3 }}-dev"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true
          - ApplyOutOfSyncOnly=true

So environments/dev/application/app-a becomes app-a-application-dev in namespace application-dev. Each env has its own values.yaml. A change under dev/ cannot touch prod, because they are different directories, different Applications, and different namespaces.

I named the ApplicationSet preproduction so the UI label is obvious, even though the git path stays environments/dev.

Platform
#

Cluster services live one level shallower: platform/<namespace>/<service>. No env segment, no -dev suffix.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: git@github.com:your-org/platform-gitops.git
        revision: main
        directories:
          - path: argocd/platform/*/*
  template:
    metadata:
      name: "{{ index .path.segments 2 }}-{{ .path.basename }}"
    spec:
      project: platform
      source:
        repoURL: git@github.com:your-org/platform-gitops.git
        targetRevision: main
        path: "{{ .path.path }}"
        helm:
          releaseName: "{{ .path.basename }}"
      destination:
        server: https://kubernetes.default.svc
        namespace: "{{ index .path.segments 2 }}"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true
          - ApplyOutOfSyncOnly=true

platform/monitoring/grafana becomes Application monitoring-grafana, release grafana, namespace monitoring.

Step 4 (What a Service Directory Looks Like)
#

Each discovered directory is a thin Helm umbrella. The chart itself lives in a Helm repo. Chart.yaml points at that remote chart and pins the version:

apiVersion: v2
name: app-a
description: Umbrella over the published app-a chart.
type: application
version: 0.1.0
dependencies:
  - name: app-a
    version: "1.2.0"
    repository: "https://charts.example.com"

Values have to be nested under the dependency name, otherwise they never reach the subchart:

app-a:
  replicaCount: 2

  image:
    repository: registry.example.com/app-a
    tag: "1.2.3"

  imagePullSecrets:
    - name: registry-credentials

  ingress:
    enabled: true
    className: traefik
    hosts:
      - host: app-a.example.com
        paths:
          - path: /
            pathType: Prefix

Dev gets its own copy of this directory under environments/dev/…, usually with replicaCount: 1 and a dev- hostname. Same remote chart, different values, different namespace.

Platform apps work the same way. Grafana, for example, depends on https://grafana.github.io/helm-charts and pins the version that’s already running so the first sync is an adoption, not a rebuild.

If you have a custom chart that doesn’t have its own Helm repo, keep it in this gitops repo (under argocd/charts/, for example) and point the umbrella at it with a local path:

dependencies:
  - name: my-custom-chart
    version: "0.1.0"
    repository: "file://../../../../charts/my-custom-chart"

The ../ depth is relative to the app directory. Same nesting rule for values: put overrides under the dependency name.

If the app needs secrets, apply them to the cluster by hand for now (kubectl apply on the Secret, or create them however you already do). In a later post I’ll show how to keep them in git and decrypt them with a custom Argo CD plugin.

Step 5 (Adding a New Application)
#

This is where the layout pays off. New image in the registry, then:

  1. Create the prod (and dev) directories with a Chart.yaml and values.yaml.
  2. If it’s a brand-new namespace, add that namespace to the matching AppProject.
  3. Commit and push.

That’s it. You don’t touch the ApplicationSets. On the next git scan, Argo CD creates the Application and syncs it.

Same story for a platform service: drop it under platform/<namespace>/<service>/ and let that ApplicationSet find it.

Step 6 (Applying and Verifying)
#

Once the repo is ready:

kubectl apply -f argocd/root.yaml

Then:

kubectl get applicationsets -n argocd
kubectl get applications -n argocd

You should see three ApplicationSets, and one Application per service directory (app-a-application-prod, app-a-application-dev, monitoring-grafana, and so on), all synced and healthy.

A Few Practical Notes
#

Keep prune off on the root app. If git is briefly empty or a path moves, you don’t want the root deleting ApplicationSets, which would then delete live apps.

Turn on selfHeal for the generated apps. If someone changes something by hand, Argo CD brings it back to git.

Pin chart versions. Especially for upstream charts. Don’t float on *. Adopt what’s already running, then bump on purpose.

New namespace = project update. The directory generator will happily create an Application targeting a namespace the AppProject does not allow. You’ll get a permission error until both destinations and the Namespace whitelist are updated.

Think about how prod should sync. Auto-sync from main keeps this example small. A stricter setup might use manual sync, a release branch, or tags.

Wrapping Up
#

The thing I like about this is that the repo structure does most of the talking. Prod and dev are separate directories, platform services aren’t pretending to be business apps, and adding something new doesn’t require editing a generator list.

I left a few topics out on purpose. SSO and tighter RBAC around who can sync what deserve their own posts, including Argo CD with Keycloak. Secrets in git will too: next time I’ll walk through a custom Argo CD plugin that decrypts them at render time. If you want to look into Keycloak now, the official Argo CD Keycloak guide is a good place to start.