TL;DR

This the second part, continuing from Part 1 . It focuses on implementing Cluster Generator in the ApplicationSet object through an example case, then on how to design the repository structure with a simple approach ✌️.

Example case

We want to deploy the staging environment on the current existing Kind Cluster (kind-infra-mgmt/in-cluster), while the production will be deployed to two clusters (c1 and c2 Clusters). Sometimes, in real cases, it is divided by “region based” such as asia-cluster, eu-cluster and etc. Depending on how the organizations groups them.

Directory structure

Here is the “simple idea” how to structure the directory ¯\_(ツ)_/¯. All example repository can be access in here .

.
├── 1-base-app/
│   ├── super-app/
│   └── another-super-app/
├── 2-mgmt/
│   ├── applicationsets/
│   │   ├── prd-appset.yaml
│   │   └── stg-appset.yaml
│   └── root-apps.yaml
└── overlays/
    ├── production/
    │   └── super-app/
    │       ├── c1-cluster/
    │       │   └── kustomization.yaml
    │       └── c2-cluster/
    │           └── kustomization.yaml
    └── staging/
        └── super-app/

Based on the ASCII tree above, let’s examine one by one the directory structure below.

• 1-base-app: It’s the all bases applications will be placed in this directory. In the current example, we will utilize a lot of Kustomize 👌.
• 2-mgmt: This directory will act as centralized “management”. It consists of how we define AppProject object, Application of applications pattern and also the ApplicationSet with the Cluster generator.
• overlays: All environment-related config for the application will be placed in here. As shown in the ASCII tree, the child of overlays/production/* will contain subdirectories (again and again 🤷) that describe cluster grouping.

The ApplicationSet

Cluster generator

Hereby the ApplicanSet with Cluster generator. Generators will be defined under .spec.generators and it will use a selector based on query filter matches env: production. So, make sure the labels have been defined on c1 and c2 clusters.

Moving to the another important spec was .spec.template this is where the application object is defined. It’s specifies how the logical naming of the application object (metadata), the project group, the sources Git repository path and branch and destination clusters.

There is go-template built-in template that helps with generating names, variables and other configurations. I haven’t tried to deep dive yet, but for Cluster generator, there are built-in templates like:

• {{.name}} = clusters name
• {{.server}} = server destination (ip or dns)

Cluster generator also having .values spec to define template variables rather than using List generator.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: production-multi-playground-super-app
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - clusters:
        selector:
          matchLabels:
            env: 'production'
        values:
          namespace: prd-super-app
  # apps object template
  template:
    metadata:
      name: 'prd-{{.name}}-super-app'
    spec:
      project: multi-playground
      source:
        repoURL: https://gitlab.com/rumble-o-bin/playground/multicluster-play.git
        targetRevision: main
        path: 'app/overlays/production/super-app/{{.name}}'
      destination:
        name: '{{.name}}'
        namespace: production
...

Git generator (staging environment)

Additionally, here’s how to deploy the staging environment while it focusing on deploying to only 1 cluster. It’s simple: just use Git generator and mapping it to overlays/staging/* and all applications inside those subdirectories will be deployed.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: staging-multi-playground
  namespace: argocd
spec:
...
  generators:
    - git:
        repoURL: https://gitlab.com/rumble-o-bin/playground/multicluster-play.git
        revision: main
        directories:
          - path: app/overlays/staging/*
  # apps object template
  template:
    metadata:
      name: 'stg-{{.path.basename}}'
    spec:
...
        path: '{{.path.path}}'
      destination:
...

Main app of apps patern

This is the last method that needs to be defined to able act as the root or main object file for grouping all the ApplicationSet that we defined above. It’s also a place to describe the AppProject object manifest.

I’m not gonna explain this all just to mention we need file like this 😄.

# root-apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  annotations:
    argocd.argoproj.io/sync-options: PruneLast=true
  # project name
  name: multi-playground
  namespace: argocd
spec:
...
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  finalizers:
  - resources-finalizer.argocd.argoproj.io
  name: multi-playground
  namespace: argocd
spec:
  # destination to mgmt argocd itself
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  project: multi-playground
  # source.path to directories of ApplicationSet
  source:
    path: app/2-mgmt/applicationsets
    repoURL: https://gitlab.com/rumble-o-bin/playground/multicluster-play.git
    targetRevision: main
  syncPolicy:
...

Push them all to Git repository, then applying with kubectl apply -f app/2-mgmt/root-apps.yaml ✌️.

Summary

Welp. At the end of writing this blog, we have learned how to simulate multi-cluster deployment locally by using Kubernetes in Docker (KinD) and vCluster as child clusters, all managed as one fleet by Argo CD.

There are many deep advanced features that can be exploring inside Aro CD itself, along with a bunch great tool stack’s in the Argo Project ecosystems. In addition, don’t forget to delete the Kind cluster after playing with the playground 😄 ✌️.

# makefile
make delete_cluster

Thank You.