Bill Gernert
CI/CD & GitOps

Argo CD

GitOps reconciliation that keeps the cluster matching git - continuously, and by force. Git is the desired state; anything else is drift to be corrected.

What it is & why I use it

Argo CD is the GitOps engine for the cluster. It continuously compares what is declared in git against what is running, and reconciles the two so the live state converges on the committed state. The repo is not a suggestion or a record after the fact - it is the control input.

I use it because it makes the cluster auditable and reproducible: every change is a commit, every rollback is a revert, and "what is supposed to be running?" has one answer that lives in version control rather than in someone's memory of what they last applied by hand.

How I use it in the lab

  • Declared apps, reconciled continuously. Cluster workloads are defined declaratively in git; Argo CD watches the repo and the cluster and keeps them in step.
  • Auto-sync with self-heal. Approved changes roll out automatically, and out-of-band drift is actively corrected back to the committed state rather than just flagged.
  • Change is a pull request. Because the repo is the control plane, a change to the running cluster is a reviewed PR - the same guardrails as any other code lands on infrastructure.
  • Visible drift and health. The sync/health view makes it obvious when something has diverged or a rollout is unhealthy, so problems surface instead of hiding.

Architecture

Diagram coming Reconcile loop: git desired state -> Argo CD diff -> sync / self-heal -> cluster, with drift corrected back to git. Scrubbed of cluster addresses and hostnames.

In action

Screenshot coming The application view showing synced/healthy status and a drift being reconciled away. Captured per the publishing checklist - hostnames and addresses redacted.

Lessons learned & gotchas

Gotcha - self-heal will undo your emergency hotfix The first time you fix something live - a quick edit straight to the cluster to stop the bleeding - and watch Argo CD calmly revert it on the next reconcile, the lesson lands hard. With self-heal on, the cluster is not the source of truth; git is. A hand-applied change is drift by definition, and drift is exactly what self-heal exists to erase. The fix is not to fight it or disable it in a panic - it is to make the change in git, even in an incident, so the emergency fix and the desired state are the same thing. GitOps means the repo is the steering wheel; touching the cluster directly is grabbing the tyre.
  • Decide what self-heal is allowed to prune. Reconciliation that removes resources not in git is powerful and unforgiving - be deliberate about scope so a reconcile never deletes something you meant to keep but forgot to declare.
  • Argo syncs manifests, not everything. It reconciles what is declared as Kubernetes state; out-of-band concerns it does not own (for example, a database schema behind an app) still need their own apply step. Do not assume a green sync means the whole world is in the desired state.
  • Drift is information, not just noise. A resource that keeps drifting is often telling you something is changing it out of band - chase the cause instead of only re-syncing the symptom.

Impact

Git
single source of truth
Auto
drift correction
Revert
= rollback

Argo CD is what makes the cluster honest: the running state cannot quietly diverge from what is committed, because divergence gets corrected on sight. The self-heal lesson is the whole philosophy in one bruise - once the repo is the control plane, the discipline is to always steer from there, especially under pressure, not to reach past it.

Code & further reading

App manifests (link coming) Argo CD docs

Published manifests are scrubbed of cluster addresses, hostnames, and any secret references before they go public.