ADR-0011 — Direct-to-storage data plane via scoped presigned URLs
Context
File bytes are large and high-volume. Proxying them through Go compute (gateway/ services) is a memory, CPU, and egress-cost disaster and couples data-plane throughput to control-plane replicas (R5/R12). The standard cloud-native answer is direct client ⇄ object-store transfer via presigned URLs. The catch: a presigned URL bypasses BitVault’s authz and audit — it is a bearer capability.
Decision
- BitVault never proxies bulk bytes. Uploads and downloads happen directly between client and object storage using presigned URLs (multipart for large files), for all providers (ADR-0005).
- Presigned URLs are issued only after an authz check (ADR-0010) and are
tightly scoped: exact object key, single method (PUT/GET), short TTL,
content-length-range, and (on upload) constrained content-type. Uploads target a staging key; the file becomes real only via the commit protocol (ADR-0004). - The control plane retains authority over metadata: access decisions, the commit/verify step, versioning, and audit all happen in BitVault; only the byte movement is delegated. Audit records the issuance of a URL (who, what, when).
- Optional CDN in front of object storage for public/download-heavy reads.
Consequences
Positive
- Data-plane throughput scales with the object store, independently of the control plane (R5) — proven by load test (09 P4).
- Minimal egress through our compute → major cost reduction (R12).
- Resumable/multipart large uploads without buffering in our services (R9).
Negative / costs
- The data plane is a deliberate, controlled bypass of the control plane — accepted explicitly. We mitigate the bearer-capability risk with tight scope + short TTL + content-length/type constraints + issuance auditing.
- Audit of actual bytes transferred is indirect (we see issuance + the commit’s HEAD verification, not every GET). Acceptable; storage-access logs can supplement.
- Clients must implement the two-step (presign → transfer → commit) protocol; the CLI and web client encapsulate it, and it’s documented for third parties.
- Browser uploads require correct CORS on the bucket (operational detail captured in runbooks).
Alternatives considered
- Proxy bytes through the API: rejected — the R5/R12 disaster; does not scale, expensive, defeats the cloud-native point.
- Long-lived/broad presigned URLs: rejected — widens the bearer-capability blast radius; we keep them single-key, single-method, short-TTL.
- Server-side streaming with auth on every chunk: rejected — reintroduces the proxy cost; presigned scoping achieves the security goal far more cheaply.