Skip to content

Infrastructure

1. Overview

Packaging and deployment for two static binaries: separate multi-stage Docker builds produce the mayu and inferplaned distroless images. The gateway Helm chart renders config into a ConfigMap and wires an optional IRSA ServiceAccount for Bedrock.

2. Components

Component Path Purpose
Dockerfile Dockerfile Multi-stage CGO_ENABLED=0 build → distroless/static:nonroot
Control-plane Dockerfile Dockerfile.inferplaned Separate static inferplaned image
Docker ignore .dockerignore Excludes tests/docs/charts from the build context
Helm chart charts/inferplane/ Deployment, Service (data+admin), ServiceAccount, ConfigMap, optional policies ConfigMap (/etc/inferplane/policies, live-reloaded — ADR-035), optional Ingress, optional PVC (ADR-023), NOTES.txt
GovernancePolicy CRD deploy/crd/ kubectl-native schema validation for inferplane.dev/v1alpha1 documents (structural schema + CEL, K8s 1.25+); controller-watch is a named follow-up (ADR-035)
Chart values charts/inferplane/values.yaml Image, replicaCount (local=1; shared Postgres supports multiple replicas), existingSecret, IRSA annotation, ingress (data/admin hosts), persistence (opt-in PVC for the key store), commented config.otel OTLP-trace example
Grafana dashboard deploy/grafana/inferplane.json 9-panel Prometheus dashboard
Product documentation mkdocs.yml, docs/assets/, scripts/docs_hooks.py MkDocs Material + static-i18n; Korean/English product guides, localized navigation/search, repository-link handling
Documentation delivery .github/workflows/docs.yml Strict build, translation coverage/freshness, and rendered-link checks on PRs; Pages deployment from main with isolated deployment permissions

3. Key Decisions

  • Build with Go 1.26.10 or newer; both Docker builders pin 1.26.10. The HTTP/2 security fixes in golang.org/x/net v0.60.0 require Go 1.26+.
  • CGO_ENABLED=0 static binary so the image can be distroless/nonroot with no libc.
  • The admin key console's static assets (internal/server/adminui/static/) ship inside the binary via go:embed — no image, chart, or build-pipeline change (ADR-001).
  • Config hot-reload (ADR-006): edit config and kill -HUP <pid> (K8s: signal PID 1 or roll the pods) to apply provider/model/pricing changes with no restart — the topology is swapped atomically, governance counters/keystore/audit persist, and a bad config rolls back. Listen addrs, TLS, drain, and team policy limits are NOT hot (restart required).
  • The chart hashes the main config into the pod template, so Helm upgrades of restart-required settings roll pods. Separately mounted policy files remain watched. Its default admin token ref reads INFERPLANE_ADMIN_TOKEN from the existing Secret; replace server.admin_auth deliberately for another setup.
  • Gateway startup probes allow ten minutes for synchronous replay before liveness begins; tune startupProbe to your retained history. Pod fsGroup defaults to 65532 for writable nonroot data volumes. Existing claims and CSI drivers must support that ownership.
  • Typed config fields reject unknown names, negative limits, invalid on_exceeded actions and out-of-range monetary caps. Inert _comment, _comment_* and _*_example annotations are allowed; map names and classifier options remain free-form. Nonzero quota.tokens_per_month is unsupported; monthly tokenQuota enforcement requires the shared Postgres profile.
  • mayu config validate --config config.json performs offline schema/secret-ref validation through LoadRaw. It does not open runtime stores, construct provider/AWS credentials or contact upstreams; a valid declaration does not establish runtime readiness. serve checks those runtime dependencies.
  • Both gateway listeners have a 10-second request-header deadline. There is no total response deadline that would cut off long-running SSE streams.
  • server.max_request_bytes (default 64 MiB) bounds one request body. server.max_inflight_request_bytes (default 0, off; otherwise at least the per-request cap) bounds the body bytes all data-plane requests hold at once. Each request reserves its Content-Length, or the whole per-request cap when the length is undeclared, until its handler returns; a stream holds it until it ends. A generation request that does not fit gets a protocol-shaped 503 with Retry-After: 2; a count request stays HTTP 200 with a local estimate of four bytes per token. Ingress holds several copies of each body (about 5.5× measured on 2026-10-27), so size the budget as (memory limit − idle RSS) / 6 and also set GOMEMLIMIT and the container or systemd memory limit. A request whose body arrives slowly or stalls keeps its reservation until the body finishes, the client gives up or the connection closes (there is no body read deadline), so size the budget at several times server.max_request_bytes: an equal value lets one stalled upload refuse every other request. Watch inferplane_inflight_refusals_total{route}.
  • Default/local mode is single-replica. ADR-046 shared Postgres mode supports multiple gateways; the chart rejects unsafe local replication. Persistent shared mode renders a StatefulSet with separate PVCs, required node anti-affinity and a disruption budget. See shared governance.
  • Persistence: local mode retains its existing Deployment/PVC behavior. Shared mode stores keys and counters in Postgres; per-replica PVCs hold independent audit segments. Do not share one audit WAL between replicas.
  • The chart references an existingSecret and never creates secrets (design §7).
  • Ingress is off by default (ingress.enabled: false); when on, the admin plane additionally requires ingress.admin.enabled: true to be routed — it carries key-issuance/governance actions, so exposing it is an explicit second opt-in, not a side effect of turning on the data-plane Ingress.
  • OTel Collector contract — five channels, two of them OTLP. A collector cannot pick up everything from one receiver, so the split is deliberate:
  • metrics — a prometheus receiver scrapes http://<svc>:9090/metrics (the Service's named admin port). The Prometheus registry (internal/metrics) is the single source of truth for metrics; there is deliberately no OTLP metric exporter, because a second SDK would double-instrument the same counters and let the two drift. gen_ai_* names follow OTel GenAI semconv naming, but the transport is Prometheus exposition, not OTLP.
  • traces — an otlp receiver on :4318 (http) or :4317 (grpc), pushed by the opt-in config.otel block (ADR-011) and by governed exporters that allow traces. Spans carry GenAI request/response attributes plus inferplane.usage.cache_{read,write_5m,write_1h}_input_tokens, inferplane.cost.amount_usd_micros / inferplane.cost.pricing_missing, and inferplane.response.partial on an interrupted stream (span status Error even though the wire status was already 200).
  • governed logs — OTLP logs (/v1/logs or the gRPC logs service) carrying audit-record copies and egress content, sent by exporters declared in config.observability.exporters to the teams/users an observability policy rule selects (governed export). Best-effort and bounded; the WAL chain stays the record of evidence.
  • observability console — the optional observability.console block names a state_file (runtime export limits that mayu writes) and a policy_file (console-owned observability rules, which must also be an entry of policies and cannot be used with control_plane). Both must be on a writable volume: the chart mounts the data volume at /var/lib/inferplane (an emptyDir unless persistence.enabled), while its policies ConfigMap mount at /etc/inferplane/policies is read-only. Both files, like the console's counters, belong to one mayu process: with more than one replica an edit applies only to the replica that served it (the status instance names it), and with the default emptyDir a replaced pod starts without them, so a pause lapses. Set persistence.enabled whenever state_file is set (the shared profile then renders one claim per replica), and keep one replica per edit target or make the edit on each replica.
  • usage windows — POST /v1alpha1/usage to inferplaned (ADR-036). This is inferplane's own protocol on its own channel; it is NOT OTLP and no collector receiver consumes it.
  • audit records — the chart default writes them to container stdout (a best-effort sink; the WAL stays on the data volume and the local analytics index is disabled). A node collector's filelog receiver tails the container log and exports each record body verbatim, never re-serialized, because the hash chain covers exact line bytes (example, collection). Give the collector separate metrics, traces and audit logs pipelines, each with a batch processor. /metrics is unauthenticated by design and must stay cluster-internal (it is on the admin port, which ingress.admin.enabled gates) — it is cardinality-bounded and carries no secret or key_id, but it is still spend data. The chart ships no ServiceMonitor/PodMonitor: those CRDs belong to the operator's monitoring stack, and the named admin port is all a scrape config needs.
  • NOTES.txt is the "easy deploy" surface: it prints the actual reachable address (Ingress host or a ready-to-paste kubectl port-forward), the first-key command, and the Claude Code env vars — so helm install alone gets an operator to working traffic without re-deriving them from values.yaml.

4. Code Pointers

  • Dockerfile — build + runtime stages
  • charts/inferplane/templates/deployment.yaml — pod spec, ports 8080/9090
  • charts/inferplane/templates/configmap.yaml — rendered config.json
  • charts/inferplane/templates/ingress.yaml — optional data/admin Ingress rules
  • charts/inferplane/templates/pvc.yaml — optional PVC for the key store (ADR-023)
  • charts/inferplane/templates/NOTES.txt — post-install quickstart

5. Cross-references

Routing rollout (ADR-043)

Upgrade all mayu instances, inferplaned, and the GovernancePolicy CRD when used before activating sensitiveData/context rules. Install routes, explicit pricing, and verified boundary/capability metadata first. Local policy is revalidated after effective topology assembly and before listening; CP ApplyWire validates targets on each data plane. Set control_plane.require_sync for privacy from first request, optionally max_policy_age for staleness. Counts stay local/200 while unready/stale. Use the isolated examples/policy-routing/governance.yaml, not the quick-start examples/policies/ directory. Shadow applies only to context preferences; privacy already enforces. No fleet HA or durability improvement is implied. See operator guide.

ADR-044 keeps policy decisions and bounded affinity state node-local. Built-in deterministic inspection has no network dependency; optional HTTP classifiers add endpoint availability and timeout dependencies, with onUninspectable applied in Enforce. Backend failover remains inside approved privacy/cost constraints. Already installed rules remain usable during a control plane outage while required authority is valid; expired hard leases still deny. ADR-045 adds opt-in Postgres global monetary authority with interchangeable control-plane replicas and private durable node journals. Its readiness endpoint checks the database; data planes retain only finite previously committed credit during outages. Deploy a replicated database and a stable control-plane endpoint. ADR-045 does not globalize local keys/rates/token quotas; ADR-046 supplies the separate synchronous shared-admission profile described above. See deployment and failure behavior.