Skip to content

Kubernetes + Helm deployment plan

Status: design only

Nothing on this page is implemented yet. The stack today is single-host docker compose (see Getting Started). This page is the plan for a Helm chart, kept in-tree so its shape can be discussed before any code exists.

Why

docker compose is the right default for one dev box — setup.sh provisions it and docker compose up -d is the whole deployment story. But the services here are plain containers, and some users already run Kubernetes: k3s on the same box, a homelab cluster, or a managed cluster for the shared pieces. A Helm chart would make the stack reproducible there without a second implementation of every service.

What maps cleanly

compose Kubernetes
service Deployment + Service
profiles enabled: flags in values.yaml
named volumes PersistentVolumeClaims
loopback debug ports ClusterIP Services (an Ingress replaces the need)
.env Secret (passwords/tokens) + ConfigMap (settings)
cloudflared tunnel the same container + token Secret, or Ingress + cert-manager
Dockerfile image built once, pushed to a registry (CI job)

What does not map (and what to do about it)

  • setup.sh provisions the host (disk layout, docker, NVIDIA driver). Cluster nodes are not managed by this repo — out of scope for the chart; the chart's README should say so.
  • DooD — the docker socket — does not belong in a pod. A pod that can drive the node's docker daemon is effectively root on that node, and unlike a single dev box there is usually no "the node is mine" argument. The chart should ship the dev services without the socket, and document the socket as an explicit opt-in only for single-node clusters you own. Portable alternatives: a dedicated build job (Kaniko/BuildKit), or no builds in cluster at all.
  • --disk / --nvidia host steps are replaced by StorageClass choices and GPU node selectors / device plugins.

Proposed chart layout

charts/ainvironment/
├── Chart.yaml
├── values.yaml               # per-service enabled flags, images, resources
├── templates/
│   ├── _helpers.tpl
│   ├── secret.yaml           # passwords (opencode, code-server, obsidian, ...)
│   ├── deployment.yaml       # one loop over the enabled services
│   ├── service.yaml
│   ├── pvc.yaml              # workspaces + per-service state
│   └── ingress.yaml          # optional; cloudflared or a standard Ingress
└── README.md

Values sketch:

services:
  code-server: { enabled: true,  port: 8080 }
  opencode:    { enabled: false, port: 4096 }
  openchamber: { enabled: false, port: 3000 }
  terminal:    { enabled: false, port: 7681 }
  obsidian:    { enabled: false, port: 3001 }
  t3code:      { enabled: false, port: 3773 }
  ollama:      { enabled: false, gpu: false }
persistence:
  workspaces:
    size: 20Gi
    storageClass: ""
ingress:
  enabled: false
  host: ainv.example.com

Phased plan

  1. Chart scaffold + code-server. The one service everyone runs: Deployment, Service, a workspaces PVC, a password Secret. helm template in CI so the chart can't drift unbuilt.
  2. opencode + openchamber. One shared password Secret; compose's depends_on has no equivalent — use readiness probes and the same OPENCODE_HOST wiring.
  3. terminal + obsidian + t3code as optional flags (loopback-only ports become ClusterIP Services; auth stays required — a Secret, or t3code's pairing links).
  4. ollama, with an optional GPU block (nodeSelector + resources.limits for nvidia.com/gpu).
  5. Ingress: a standard Ingress + cert-manager, or keep cloudflared as a Deployment with TUNNEL_TOKEN from a Secret. (The cloudflared route rows carry over unchanged; only the targets change from service names to <release>-<service> names.)
  6. Publish the dev image to the project's container registry from CI, so the chart has a stable image to pull.

Open questions

  • Target cluster: k3s on the same box, or a "real" cluster? This decides whether the DooD socket gets a documented escape hatch at all.
  • One chart or several? Proposal: one ainvironment chart — the services are only useful together, and profiles are just enabled flags.
  • Workspace PVC access mode: ReadWriteOnce (simple, single node) vs ReadWriteMany (NFS/CephFS, services on different nodes). Proposal: RWO first, document RWX.
  • Secrets: plain Secret values first; SOPS / External Secrets as the next step, documented but not required.