Dev image automation

info

This runbook explains how Flux image automation keeps the Blaster dev environment pointing at the latest dev-* image tag pushed by GitLab CI. It covers registry access, ImageRepository, ImagePolicy and ImageUpdateAutomation resources, plus how they interact with the app repo Deployment.

Blaster GitOps series

1. Context and goals

GitLab CI builds Blaster images on develop and main using Kaniko.

Dev images are tagged:

registry.reids.net.au/games/blaster:dev-YYYYMMDD.N

Prod images are tagged:

registry.reids.net.au/games/blaster:prod-YYYYMMDD.N

Goal for dev:
- Flux discovers all dev-* tags in the registry.
- A policy picks the latest tag.
- Flux commits the new tag into k8s/dev/50-app-deployment.yaml in the app repo.
- The blaster-dev Kustomization applies the change to the cluster.

This document focuses on dev (blaster-dev). Prod can be configured the same way with a prod policy.

For Dockerfile and CI pipeline details see Dockerfile & GitLab CI.
For Flux sources and Kustomizations see Flux sources & Kustomizations.
For app manifests see App k8s manifests.

2. Registry access for image scanning

Flux needs to authenticate to the GitLab Container Registry in order to scan tags.

2.1 Create a GitLab deploy token

In the games/blaster project:

Go to Settings → Repository → Deploy tokens.
Create a token:
- Name: blaster-dev-token
- Username: for example blaster-dev (or leave blank for gitlab+deploy-token-...)
- Scopes:
  - read_registry
  - read_repository
Store the username and token value somewhere secure.

This token will be used in a Kubernetes Secret of type kubernetes.io/dockerconfigjson.

2.2 Registry Secret for Flux (flux-system namespace)

On your workstation, inside the flux-config repo:

cd ~/Projects/flux-config

kubectl -n flux-system create secret docker-registry blaster-dev-registry   --docker-server=registry.reids.net.au   --docker-username='blaster-dev'   --docker-password='REPLACE-ME'   --docker-email='andrew@reids.net.au'   --dry-run=client -o yaml   > clusters/my-cluster/flux-system/secrets/blaster-dev-registry.yaml

Then encrypt with SOPS:

sops -e -i clusters/my-cluster/flux-system/secrets/blaster-dev-registry.yaml

Infra repo .sops.yaml (simplified):

# .sops.yaml
creation_rules:
  - path_regex: 'clusters/my-cluster/.*/secrets/.*\.yaml$'
    encrypted_regex: '^(data|stringData)$'
    age: ['AGE_PUBLIC_KEY_HERE']

Finally, ensure the secret is included in the Flux-system Kustomization:

# clusters/my-cluster/flux-system/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
  - ./secrets/blaster-dev-registry.yaml

After committing and pushing, reconcile:

git add .
git commit -m "Add blaster dev registry secret"
git push

flux reconcile kustomization flux-system -n flux-system --with-source
kubectl -n flux-system get secret blaster-dev-registry

3. ImageRepository and ImagePolicy

Image automation uses two CRDs:

ImageRepository to fetch tags from a registry.
ImagePolicy to select the winning tag.

3.1 ImageRepository

File: clusters/my-cluster/blaster/dev/20-blaster-images-dev.yaml

# clusters/my-cluster/blaster/dev/20-blaster-images-dev.yaml
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageRepository
metadata:
  name: blaster-dev-repo
  namespace: flux-system
spec:
  image: registry.reids.net.au/games/blaster
  interval: 1m
  secretRef:
    name: blaster-dev-registry

Key points:

image is the repository, not a full tag.
secretRef.name matches the registry Secret created earlier.
interval: 1m means Flux will rescan for new tags roughly every minute.

Verify:

flux get image repository -n flux-system
kubectl -n flux-system get imagerepository blaster-dev-repo -o yaml | sed -n '1,80p'

You should see READY=True and a list of discovered tags in the status section.

3.2 ImagePolicy

In the same file, define an ImagePolicy:

---
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImagePolicy
metadata:
  name: blaster-dev-policy
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: blaster-dev-repo
  filterTags:
    pattern: '^dev-(?P<ts>[0-9]{8}\.[0-9]+)$'
    extract: '$ts'
  policy:
    numerical:
      order: asc

Explanation:

filterTags.pattern matches tags that look like dev-20251115.51.
The named group (?P<ts>...) captures the timestamp-style suffix.
extract: '$ts' tells Flux to compare based on that numeric string.
policy.numerical.order: asc means the highest timestamp becomes the latest tag.

Check the policy:

flux get image policy -n flux-system
kubectl -n flux-system get imagepolicy blaster-dev-policy -o yaml | sed -n '1,80p'

In the status you should see something like:

latestImage: registry.reids.net.au/games/blaster:dev-20251115.51

4. Marking the Deployment for updates

Flux needs a hint inside the app repo manifest to know which field to update.

In the Blaster app repo, Dev Deployment:

File: games/blaster/k8s/dev/50-app-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: blaster-app
  namespace: blaster-dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: blaster-app
  template:
    metadata:
      labels:
        app: blaster-app
    spec:
      imagePullSecrets:
        - name: blaster-dev-registry
      containers:
        - name: blaster
          image: registry.reids.net.au/games/blaster:dev-20251115.42 # {"$imagepolicy": "flux-system:blaster-dev-policy"}
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 3000
              name: http
          ...

The important bit is the JSON-style comment:

# {"$imagepolicy": "flux-system:blaster-dev-policy"}

This tells Flux that:

It should use the blaster-dev-policy in namespace flux-system.
It should update the image: field on that line when the winning tag changes.

You only add this comment once. After that, Flux will keep editing the tag.

Commit this to the app repo on develop:

git add k8s/dev/50-app-deployment.yaml
git commit -m "Mark dev deployment for image automation"
git push

5. ImageUpdateAutomation

ImageUpdateAutomation performs the Git operations:

Checks out the app repo.
Applies the policy-driven image update.
Commits and pushes the changes to the branch.

File: clusters/my-cluster/blaster/dev/30-image-automation.yaml

# clusters/my-cluster/blaster/dev/30-image-automation.yaml
apiVersion: image.toolkit.fluxcd.io/v1beta2
kind: ImageUpdateAutomation
metadata:
  name: blaster-dev-automation
  namespace: flux-system
spec:
  interval: 1m
  sourceRef:
    kind: GitRepository
    name: blaster-dev
  git:
    checkout:
      ref:
        branch: develop
    commit:
      author:
        name: FluxCD
        email: andrew@reids.net.au
      messageTemplate: '{{range .Changed.Changes}}{{print .OldValue}} -> {{println .NewValue}}{{end}} [skip ci]'
    push:
      branch: develop
  update:
    strategy: Setters
    path: ./k8s/dev

Fields to watch:

sourceRef.name: blaster-dev
This must match the GitRepository that tracks the games/blaster repo on the develop branch.
checkout.ref.branch: develop and push.branch: develop
Means automation happens directly on the develop branch.
messageTemplate
Includes [skip ci] to avoid triggering an infinite loop of pipelines when Flux commits a new image tag.
update.strategy: Setters and path: ./k8s/dev
Used in combination with the # {"$imagepolicy": ...} comment to locate the field to update.

After adding this file:

cd ~/Projects/flux-config
git add clusters/my-cluster/blaster/dev/30-image-automation.yaml
git commit -m "Add image automation for blaster dev"
git push

flux reconcile kustomization blaster-dev -n flux-system --with-source

Verify:

flux get image update -n flux-system
kubectl -n flux-system get imageupdateautomation blaster-dev-automation -o yaml | sed -n '1,80p'

You should see READY=True and a recent lastRunTime.

6. Giving Flux write access to the app repo

For ImageUpdateAutomation to push commits, the SSH key used by the GitRepository blaster-dev must have write access in GitLab.

Identify the secret used:

clusters/my-cluster/blaster/dev/source.yaml:

spec:
  url: ssh://git-ssh.reids.net.au/games/blaster.git
  secretRef:
    name: flux-ssh-auth

Inspect the secret:

kubectl -n flux-system get secret flux-ssh-auth -o yaml

Look for:

data:
  identity: <base64-encoded-private-key>
  identity.pub: <base64-encoded-public-key>

Decode the public key:

kubectl -n flux-system get secret flux-ssh-auth      -o jsonpath='{.data.identity\.pub}' | base64 -d

In GitLab for games/blaster:
- Go to Settings → Repository → Deploy keys.
- Find the matching key.
- Enable Write access allowed.
- Save.

warning

Any process with this key can push to that repo. Keep it restricted to Flux and do not reuse it casually.

7. End-to-end flow for dev images

Once configuration is complete, this is what happens when you push code to develop:

GitLab CI on develop:
- Runs lint and tests.
- Builds an image with Kaniko, tagging it as:
```
registry.reids.net.au/games/blaster:dev-YYYYMMDD.N
```
Flux ImageRepository:
- Notices the new tag during the next scan.
- Records it in the blaster-dev-repo status.
Flux ImagePolicy:
- Evaluates all tags matching the dev-... pattern.
- Picks the latest tag as latestImage.
ImageUpdateAutomation:
- Detects that the policy resolved to a new tag.
- Checks out games/blaster at develop.
- Updates the image field marked with {"$imagepolicy": "flux-system:blaster-dev-policy"} in k8s/dev/50-app-deployment.yaml.
- Commits with a message that includes [skip ci].
- Pushes back to develop.
blaster-dev Kustomization:
- Sees a new revision of develop.
- Applies the updated k8s/dev manifests.
- The Deployment in blaster-dev namespace rolls out the new image.

You can watch this in action by:

Inspecting the policy and update status:

flux get image repository -n flux-system
flux get image policy -n flux-system
flux get image update -n flux-system

Checking the image in the Deployment:

kubectl -n blaster-dev get deploy blaster-app -o jsonpath='{.spec.template.spec.containers[0].image}'
echo

Looking at the latest Git commit on develop for an auto-generated [skip ci] message.

8. Verification checklist

If all of these are true, Blaster dev image automation is working and future dev images will roll out to the cluster with no manual tag edits.

Blaster GitOps series​

1. Context and goals​

2. Registry access for image scanning​

2.1 Create a GitLab deploy token​

2.2 Registry Secret for Flux (flux-system namespace)​

3. ImageRepository and ImagePolicy​

3.1 ImageRepository​

3.2 ImagePolicy​

4. Marking the Deployment for updates​

5. ImageUpdateAutomation​

6. Giving Flux write access to the app repo​

7. End-to-end flow for dev images​

8. Verification checklist​