Skip to main content

Kubernetes deployment

Kubernetes notes for Stage 1.

Environments​

  • development (local): Kubespray (Ansible) + local Postgres + local storage (NAS).
  • production (bare-metal cluster): Kubespray (Ansible) + external NAS (NFS) for PVCs

Namespaces​

  • fitness-dev.
  • fitness-prod.

Core Manifests (per env)​

  • deployment.yaml — API + workers (separate deployments).
  • service.yaml — ClusterIP for API.
  • ingress.yaml — Ingress (nginx) + TLS (external certficate manually installed).
  • configmap.yaml — non-secret config (feature flags, log levels).
  • secret.yaml — Strava keys, encryption keys (mounted).
  • postgres.yaml — StatefulSet + PVC (or external Postgres).
  • cronjobs.yaml — backups, analytics refresh if needed.
  • networkpolicy.yaml — minimal egress (Strava), restrict ingress.

GitOps with FluxCD​

  • GitLab is the source of truth.
  • Flux Kustomization per env/namespace.
  • Image automation: update tags on merge → rollout.
  • Rollback = git revert.

Secrets​

  • Store as SOPS-encrypted YAML.
  • Mounted as volumes/env at runtime; never committed plaintext.

Storage​

  • Thumbnails & exports: object storage path (NFS or S3-compatible).
  • DB: Postgres PVC with regular backups (see below).

Backups​

  • Nightly pg_dump CronJob → object storage (date-stamped).
  • Separate restore.md with tested steps.
  • Verify restore weekly.

Health and observability​

  • Readiness/liveness probes for API + workers.
  • Structured logs with req_id.
  • Minimal dashboard: ingest → worker → DB metrics.
  • Alert on repeated thumbnail/job failures.

Ingress and domains​

  • fit.muppit.coach → prod ingress.
  • dev.fit.muppit.coach → dev ingress.
  • ngrok stays for local OAuth testing until ingress rollout is stable.

Cost guardrails​

  • Prefer small nodes; limit replicas until needed.
  • Offload big blobs to object storage.
  • Avoid managed cloud unless there’s a clear benefit.