Can we run Kubernetes with no hyperscaler under it?
Yes. I've built it from empty machines to production in under 50 working days, for a company with no platform team. The cluster is the easy part. What takes the work is everything a cloud gives you for free: storage, secrets, load balancing, delivery and monitoring.
What you’re really asking for
A cluster is a day’s work. A platform is the cluster plus the things around it:
- Storage that survives a dead machine.
- Secrets that exist before the cluster does.
- Delivery that nobody does by hand.
- Monitoring that isn’t on the thing it watches.
- A way back when a site is gone.
What I’d choose, and why
- Talos, RKE2 or k0s for Kubernetes, depending on who’ll run it after me.
- Cilium for networking.
- Ceph for storage once there are enough machines. On 3 nodes it’s too heavy, and I’d start smaller.
- Vault outside the cluster. Kubernetes needs secrets to start.
- Flux for delivery. If it isn’t in git, it isn’t deployed.
What it takes
Weeks, not quarters. And a plan for who runs it after me, because I build these to be handed over.
What I won’t do
Carry the pager for a platform I didn’t build, without a paid handover first.