Migrating a Helm-based media stack from imperative scripts to Flux GitOps. Same charts, same values, automated delivery.

“Git push should deploy, right?” - Yes, and it can actually work that way

Dev vs Production - Manual deploy.sh vs GitOps

This builds on the media stack post. The stack should already be designed (or running). This article replaces the deploy.sh approach with Flux.


Problem

I ran deploy.sh for six months. It worked. Then problems crept in:

I kubectl edit-ed the Plex deployment to test a resource limit. Forgot about it. Three weeks later, ran deploy.sh for a Sonarr upgrade. The script overwrote the deployment with values from Git. My test config - gone. Plex OOMKilled because the limits I’d set weren’t in the repo. Lost 20 minutes figuring out what broke.

Accidentally deleted the Traefik middleware ConfigMap while cleaning up. Didn’t notice for two days. HTTPS redirects broken. Nothing recreated it automatically. I only found out when someone tried to access a service over HTTP.

Made a config change. Pushed to Git. Remembered 3 hours later I never actually ran the script. Overseerr was down. Family complained. “I thought you fixed that?”

The script works. Until you forget to run it. Or run it when you shouldn’t. Or make manual changes and forget what you did.

“Manual operations are single points of failure. The human who ‘knows the process’ is a bus factor of one.” - Automation principle

Solution

“GitOps: Because ‘it worked when I ran it from my laptop’ is not a deployment strategy.” - Platform Engineering Truth

“I aim to misbehave.” - Mal, Firefly. With Flux, you push to Git and the cluster behaves. No manual drift, no forgotten deploys. Flux watches a Git repo and continuously reconciles the cluster to match it. Push a change, Flux applies it. Delete a resource manually, Flux recreates it. Drift is corrected automatically.

Full source: k8s-gitops

ℹ️ Info

Flux runs entirely in-cluster as a set of controllers. No external server, no UI to host, no database. It adds about 200 MB of RAM to the cluster.


What Changes (and What Doesn’t)

Before (deploy.sh) After (Flux)
helm install metallb ... HelmRelease CRD in Git
helm install sonarr bjw-s/app-template -f values.yaml HelmRelease with values inline
kubectl apply -f metallb-config.yaml Same YAML, same path, Flux applies it
Run script manually after every change Push to Git, Flux reconciles
Script handles ordering with sequential execution Flux Kustomization with dependsOn

Nothing about your actual configuration changes. Same Helm charts, same values, same YAML. You’re just replacing “I run this script” with “Flux watches Git and runs it for me.”

The hard part was designing the stack. GitOps just makes it reliable.

“Git as source of truth means: diff before apply, rollback is revert, audit trail is automatic.” - GitOps value proposition


Repo Structure

k8s-gitops/
├── clusters/
│   └── homelab/
│       ├── flux-system/           # Auto-generated by flux bootstrap
│       ├── infrastructure.yaml    # Flux Kustomization → infrastructure/
│       └── apps.yaml              # Flux Kustomization → apps/
├── infrastructure/
│   ├── kustomization.yaml
│   ├── sources/
│   │   └── helm-repositories.yaml
│   ├── metallb/
│   │   ├── namespace.yaml
│   │   ├── helmrelease.yaml
│   │   └── config.yaml
│   ├── nfs-csi/
│   │   ├── helmrelease.yaml
│   │   └── storageclass.yaml
│   └── traefik/
│       ├── namespace.yaml
│       └── helmrelease.yaml
└── apps/
    ├── kustomization.yaml
    ├── media/
    │   ├── namespace.yaml
    │   ├── storage.yaml
    │   ├── plex.yaml
    │   ├── sonarr.yaml
    │   ├── ...
    │   └── tautulli.yaml
    └── dashboard/
        └── homepage.yaml

Two layers, two Flux Kustomization objects. Infrastructure must be healthy before apps deploy.


Key Concepts

HelmRepository

Tells Flux where to find charts. Replaces helm repo add:

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: bjw-s
  namespace: flux-system
spec:
  interval: 24h
  url: https://bjw-s-labs.github.io/helm-charts/

Flux refreshes the index every 24 hours.

HelmRelease

Replaces helm install / helm upgrade. The values section is the exact same YAML you’d put in a values.yaml file:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: sonarr
  namespace: media
spec:
  interval: 30m
  chart:
    spec:
      chart: app-template
      sourceRef:
        kind: HelmRepository
        name: bjw-s
        namespace: flux-system
  install:
    remediation:
      retries: 3
  upgrade:
    remediation:
      retries: 3
  values:
    controllers:
      sonarr:
        strategy: Recreate
        containers:
          app:
            image:
              repository: linuxserver/sonarr
              tag: latest
            env:
              PUID: "1000"
              PGID: "1000"
              TZ: "America/New_York"
            resources:
              requests:
                cpu: 100m
                memory: 256Mi
              limits:
                cpu: 500m
                memory: 512Mi
    service:
      app:
        controller: sonarr
        ports:
          http:
            port: 8989
    ingress:
      app:
        className: traefik
        hosts:
          - host: sonarr.media.lan
            paths:
              - path: /
                service:
                  identifier: app
                  port: http
    persistence:
      config:
        type: persistentVolumeClaim
        accessMode: ReadWriteOnce
        size: 2Gi
        storageClass: nfs-appdata
        globalMounts:
          - path: /config
      data:
        type: persistentVolumeClaim
        existingClaim: media-data
        globalMounts:
          - path: /data

Every 30 minutes, Flux checks if the cluster state matches this spec. If not, it reconciles. If the install fails, it retries 3 times.

Flux Kustomization (Dependency Ordering)

The infrastructure Kustomization deploys and waits for health checks before apps starts:

# clusters/homelab/infrastructure.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure
  prune: true
  wait: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: metallb-controller
      namespace: metallb-system
    - apiVersion: apps/v1
      kind: Deployment
      name: traefik
      namespace: traefik
# clusters/homelab/apps.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps
  prune: true
  wait: true
  dependsOn:
    - name: infrastructure

This replaces the sequential execution in deploy.sh. Flux handles the ordering declaratively.

💡 Tip

prune: true means Flux deletes resources that are removed from Git. If you delete apps/media/tautulli.yaml and push, Flux removes Tautulli from the cluster. Without pruning, orphaned resources accumulate.


Bootstrap

Install the Flux CLI

curl -s https://fluxcd.io/install.sh | sudo bash

Run Bootstrap

export GITHUB_TOKEN=<your-personal-access-token>
export GITHUB_USER=<your-username>

flux bootstrap github \
    --owner=$GITHUB_USER \
    --repository=k8s-gitops \
    --branch=main \
    --path=clusters/homelab \
    --personal

This:

  1. Installs Flux controllers into the flux-system namespace
  2. Creates a GitRepository source pointing at the repo
  3. Creates a root Kustomization that syncs clusters/homelab/
  4. Commits the Flux manifests back to the repo (the flux-system/ directory)
💡 Tip

The GitHub token needs repo scope. Create one at github.com/settings/tokens. Flux uses it once during bootstrap to set up a deploy key, then the token can be revoked.

Verify

flux check

flux get kustomizations
# NAME             READY   STATUS
# flux-system      True    Applied revision: main@sha1:abc123
# infrastructure   True    Applied revision: main@sha1:abc123
# apps             True    Applied revision: main@sha1:abc123

flux get helmreleases -A
# NAMESPACE       NAME            READY   STATUS
# metallb-system  metallb         True    Helm install succeeded
# kube-system     csi-driver-nfs  True    Helm install succeeded
# traefik         traefik         True    Helm install succeeded
# media           plex            True    Helm install succeeded
# media           sonarr          True    Helm install succeeded
# ...

Day-to-Day Workflow

“Did I run that helm upgrade or just think about running it?” - Problems you no longer have

Change a Value

Edit the HelmRelease, push to main:

# Edit apps/media/sonarr.yaml - change resource limits, image tag, etc.
git add -A && git commit -m "bump sonarr memory limit" && git push

Flux reconciles within 30 minutes. To apply immediately:

flux reconcile kustomization apps --with-source

Add a New App

Create a new HelmRelease file, add it to apps/kustomization.yaml, push. Flux deploys it.

Remove an App

Delete the HelmRelease file, remove it from apps/kustomization.yaml, push. Flux prunes it.

Suspend and Resume

“Production is the best test environment. Staging is the place where you realize this.” - DevOps Reality Check

Temporarily stop Flux from managing an app (useful for debugging):

flux suspend helmrelease sonarr -n media
# Make manual changes, debug, etc.
flux resume helmrelease sonarr -n media

Check What Flux Sees

# All Kustomizations
flux get kustomizations

# All HelmReleases
flux get helmreleases -A

# Events for a specific release
flux events --for HelmRelease/sonarr -n media

# Logs
flux logs --kind=HelmRelease --name=sonarr -n media

Migration Checklist

If you already have the media stack running from deploy.sh, Flux can adopt the existing Helm releases. The key is that the HelmRelease names and namespaces match the existing helm install names.

Existing release HelmRelease metadata.name Namespace
metallb metallb metallb-system
csi-driver-nfs csi-driver-nfs kube-system
traefik traefik traefik
plex plex media
sonarr sonarr media
media

When Flux finds an existing Helm release with the same name, it adopts it rather than creating a new one. No downtime, no re-creation.

⚠️ Warning

Make sure the values in your HelmRelease match what’s currently deployed. If Flux detects a diff, it will upgrade the release to match the Git state. Run helm get values <release> -n <namespace> to verify before bootstrapping.


Common Issues

Symptom Cause Fix
Kustomization stuck Not Ready Dependency not healthy flux get kustomizations, check health checks
HelmRelease install retries exhausted Chart values error or missing dependency flux events --for HelmRelease/<name>, check values
MetalLB config no matches for kind "IPAddressPool" CRDs not installed yet Infrastructure health check should prevent this. Check MetalLB HelmRelease.
Changes not applying Flux hasn’t reconciled yet flux reconcile source git flux-system
prune deleted something unexpectedly Resource removed from kustomization.yaml Add it back, push. Or use flux suspend before making structural changes.

References