03 — Environment Strategy
Task 3: design environment strategy. The ladder a change climbs from a laptop to production, the config that differs per rung, and how artifacts are promoted, not rebuilt.
1. The environments
| Env | Purpose | Deploys from | Auto? | Data |
|---|---|---|---|---|
| local | inner-loop dev | Compose (ADR-0012) | — | throwaway |
| dev | continuous integration of main |
latest main image digest |
auto | seeded/synthetic |
preview (pr-<n>) |
review a PR in isolation | the PR’s image | auto (on PR) | seeded, ephemeral |
| staging | prod-like release validation (e2e, load, soak) | release-candidate digest | auto on RC | synthetic, prod-shaped |
| prod | live traffic | promoted digest, canary (04) | gated | real (never copied down) |
2. Config per environment: same chart, different values
One Helm chart (05); per-environment values overlays in the GitOps repo. What legitimately differs:
| Differs per env | Example |
|---|---|
| Scale | replicas, HPA min/max, resource requests |
| Dependency tier | lite/standard/full (ADR-0012) |
| Endpoints | DB host, object-storage bucket, OIDC issuer |
| Feature flags | enable in-progress features in dev/staging first |
| Secrets | per-env secret store / KMS key (09) |
| Rollout aggressiveness | dev = fast rolling; prod = slow canary + analysis |
What must not differ: the image (same digest), the chart templates, and the application code path. Divergence beyond values is a smell (12-factor parity).
3. Promotion: build once, promote the digest
flowchart LR
classDef ci fill:#fde68a,stroke:#b45309,color:#111827;
classDef e fill:#bbf7d0,stroke:#15803d,color:#111827;
classDef g fill:#fbcfe8,stroke:#be185d,color:#111827;
build["CI builds image once → digest D (signed, SBOM)"]:::ci
build --> dev["dev (auto): values-dev pins D"]:::e
dev -->|tests/e2e green| rc["staging: PR pins D as RC"]:::g
rc -->|soak + load + manual approve| prod["prod: PR pins D → canary"]:::g
prod --> done["stable"]:::e
- The same digest
Dflows dev→staging→prod; nothing is rebuilt per environment, so “works in staging” means those exact bytes work (ADR-0034). - Promotion is a PR in the GitOps repo moving
Dfrom one env’s values to the next — reviewable, auditable, revertable. - Gates: auto-promote dev→staging on green CI; manual approval (+ green staging soak/load) for staging→prod (08).
4. Ephemeral preview environments
- On PR open, an ArgoCD ApplicationSet PR generator materializes a
pr-<n>namespace with the PR’s image + seeded data; the PR gets a URL. - On merge/close, it is torn down (and a TTL reaps stragglers) — cost-bounded by quotas (02).
- Previews use the
lite/standardtier (cheap deps) and synthetic data — never prod data.
5. Data handling across environments
- Prod data is never copied to lower environments (privacy/compliance, ADR-0007). Lower envs use synthetic, prod-shaped datasets; if real-derived data is ever needed, it is anonymized.
- Staging mirrors prod topology and scale-shape (not data) so load/soak tests are meaningful.
6. Tradeoffs / Alternatives / Scaling
Tradeoffs. More environments = more confidence but more cost/ops. We keep a tight ladder (dev/staging/prod + ephemeral previews) rather than many long-lived stages; previews give per-change isolation without permanent cost.
Alternatives considered.
- Branch-per-environment (a
staging/prodGit branch): a well-known anti-pattern — drift, painful cherry-picks, merge hell. We use directories + values per env in one branch of the GitOps repo, promoting by PR (ADR-0034). Rejected. - No staging (dev→prod): faster but unsafe for a data-custody product; staging soak catches migration/perf regressions.
- Permanent per-developer environments: costly; ephemeral PR previews cover the need.
Scaling concerns.
- Preview env cost/sprawl → TTL + quotas + reap-on-close; cheap dep tiers.
- Config drift between envs → enforced parity (only values differ), diffed in PRs; ArgoCD shows live-vs-desired drift (06).
- Many services × envs → ApplicationSet generators render the matrix from one manifest (06).
References
- ArgoCD ApplicationSet PR generator: https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Generators-Pull-Request/
- 12-factor (config, dev/prod parity): https://12factor.net/