AI Infra
0%
Summary

Summary

AuthorChangkun Ou
Reading time~3 min

Practice begins with a user promise, not a model name. A workload contract states the tasks and population in scope, the accepted outcome, the hard constraints, and the limits on quality, latency, availability, cost, and risk. The candidate is then a versioned served system: weights or a hosted model revision together with its prompt, tools, retrieval, policies, runtime, routes, and settings. Whether the workload needs a cloud service, an on-device path, a change to the weights, or a combination is a consequence of that contract. Each option must be evaluated against the same workload rather than justified by a leaderboard, parameter count, or lower unit price.

The choice becomes real through complete release units. A versioned serving contract binds runtime and compute to request semantics. A versioned device deployment adds export, quantization, supported hardware, operating state, and delivery. An adaptation release connects a behavior claim to governed data, a reproducible run, evaluation, and a reversible artifact. An agent release puts a trusted controller around model proposals, tools, durable state, authority, and effects. A retrieval release publishes a corpus and an authorized query path with lineage, freshness, deletion, and citation evidence. None of these units is complete merely because its central component runs.

Composition begins with a system fingerprint for every behavior-changing surface and an immutable identity for every released artifact. Each connection then needs a boundary contract for meaning, authorization, data class, deadline, retry ownership, idempotency, cancellation, backpressure, evidence, and recovery. Deployment is a controlled state transition, not a file copy. The team proves coexistence, runs the candidate in shadow when appropriate, limits it to a bounded canary, and widens exposure only when evidence supports the next decision. Desired state, observed state, assignment, and migration state remain distinct, and rollback restores the complete last-known-good system rather than changing only the model route.

Once the release is live, reliability is defined by a user-visible outcome, not by byte-for-byte repetition or an HTTP success code. Directly observed events can support an SLI and SLO for availability, latency, policy, freshness, or effect correctness. Semantic quality usually needs a separate probability sample, versioned rubric, calibrated judge or human review, label deadline, and coverage report. Human oversight is part of the runtime only when approval is bound to the exact effect and rechecked at commit. The events produced by that loop are evidence, not truth: a product event is not automatically a label. Collection needs a declared purpose, minimum necessary content, use authority, sampling record, partition, and deletion lineage before it can become a versioned data product.

Operation also makes scarcity and failure explicit. Cost per accepted task includes original attempts, retries, fallbacks, retrieval, tools, judging, and review. The runtime must estimate, reserve, admit, meter, and reconcile spend before a delayed billing report arrives. A qualified fallback still has to satisfy the task's security, quality, latency, and tenancy requirements. Tenant isolation is enforced separately across capacity, caches, indexes, credentials, tools, egress, evidence, review, and billing. Unknown identity, policy, effect, usage, or commit state is handled as a named state, not silently converted into success. Incidents therefore have owners, clocks, containment authority, preserved evidence, recovery criteria, and corrective actions that are verified or recorded as accepted risk.

An operating contract gathers those decisions into one versioned, enforceable artifact. It connects each promise to its measurement, each budget to its admission control, each authority to an enforcement point, each tenant boundary to a negative test, and each incident decision to evidence. Its release record identifies the active system, scope, owners, objectives, exceptions, validation, rollout, rollback, residual risks, and next review. This is the practical meaning of dependable infrastructure: not a system that never changes or fails, but one whose promises remain visible, bounded, testable, and owned after the system changes and after it fails.

Comments

Log in to comment