Skip to main content

Foundation and overview

info

I prefer operating Kubernetes on-premises because it gives me full control over the environment and, with open-source choices, it’s very low-cost to run.

Reasons to operate on-premises​

The practitioner’s case for building Kubernetes (K8s) clusters on-premises rather than using a managed cloud service.

Consider the following to determine if building on-premises fits your use case:

  • Data locality and egress economics. If most data is created or consumed locally, keeping compute beside it avoids egress fees and long transfer times. Think plants, labs, media, machine-learning feature stores, CCTV, CAD, GIS, and backups.
  • Low latency and deterministic performance. Real-time or near-real-time systems benefit from short hops, predictable jitter, and no dependency on a wide area network (WAN). Useful for operational technology (OT), trading, telecoms, medical devices, and human–machine interface (HMI) control loops.
  • Sovereignty, residency, and assurance. You need strict control of where data sits, who can touch the hardware, and how the stack is patched and audited. Air-gapped or disconnected modes are easier to guarantee on-premises.
  • Specialised hardware. You choose exact GPUs, FPGAs, SmartNICs, NVMe, remote direct memory access (RDMA), time-sync hardware, and kernel drivers. You can align CUDA, drivers, and runtimes, and use multi-instance GPU (MIG) or time-slicing without cloud SKU limits.
  • Resilience to internet failure. Sites must keep operating during WAN outages. Local clusters keep plant, point-of-sale, clinical, or branch systems up when the link is down.
  • Legacy and non-Internet protocols. Some workloads rely on multicast, broadcast, serial, or ICS protocols that do not traverse cloud networks cleanly or safely.
  • Cost predictability for steady-state. Long-lived, stable workloads can reach a lower TCO when capital expenditure (capex) is amortised and you manage power, cooling, and refresh cycles. No hourly control-plane fees or per-GB egress surprises.
  • Security posture you control. Private registries, software bill of materials (SBOM) and signature verification, admission policies, bastions, offline patch repositories, and full audit trails from firmware upwards.
  • Vendor portability. Avoiding cloud-specific IAM, load balancers, and service meshes preserves clean exit options and eases multi-site portability.

Reasons to not operate on-premises​

  • Highly bursty or experiment-heavy compute where elasticity dominates.
  • Heavy use of managed data services where the platform is not the bottleneck.
  • Global distribution and anycast edge where proximity to many users matters more than proximity to your data.

Common arguments and responses​

  • “K8s is cloud-only.” Kubernetes was designed to be portable. It runs on laptops, bare metal, and virtualised hosts just as well as in cloud regions. I choose the location that fits data, latency, and control.
  • “Too complex to run.” Unbounded choice is complex. A prescriptive stack, together with GitOps, removes most of the moving parts you touch day-to-day.
  • “Overkill for small teams.” When you need self-healing, rollouts, RBAC, policies, and storage orchestration, Kubernetes pays for itself even on a single-node cluster.
  • “Cloud is always cheaper.” Not when egress is high, data is local, or uptime must survive a WAN outage. On-premises removes egress surprises and gives predictable steady-state costs.
  • “We’ll lose speed.” A templated cluster, golden images, and GitOps mean new sites and environments are repeatable. Speed comes from templates and runbooks, not from where the servers live.