A policy statement such as 'the model must be safe enough' cannot govern a release. A usable gate names the hazard, target population, test distribution, metric, threshold, uncertainty rule, evaluator, decision owner, and required action after failure. It also distinguishes a model evaluation from a system evaluation: prompts, retrieval, tools, permissions, interface design, and human review can change real-world behavior even when model weights are unchanged.
Suppose an evaluation observes $k$ policy-relevant failures in $n$ approximately independent trials. The point estimate $k/n$ is incomplete, especially when failures are rare. With prior parameters $a$ and $b$, a binomial probability model gives $$p\mid k,n\sim\operatorname{Beta}(a+k,b+n-k).$$ The prior must be disclosed and sensitivity-tested. Correlated prompts, duplicated benchmarks, and evaluator disagreement reduce the effective sample size, so a large row count does not automatically imply strong evidence.
Write the gate against an uncertainty bound. If $\tau$ is the maximum tolerated failure probability, one rule is $$Q_{1-\alpha}(p\mid k,n)\le\tau,$$ where $Q_{1-\alpha}$ is an upper posterior quantile and $\alpha$ is the accepted error probability. This rule asks whether even a conservative plausible failure rate is below the limit. The threshold and confidence level should be set from the consequence of error, not adjusted after seeing results.
Most releases have multiple critical metrics. Let $m_j$ be the measured risk for requirement $j$ and $\tau_j$ its limit. A non-compensatory gate can be written as $$G=\mathbf{1}[m_j\le\tau_j\ \text{for every critical }j].$$ This prevents excellent performance on a low-consequence metric from averaging away a serious security or rights failure. A failed critical gate may lead to stop, redesign, restricted access, or a documented compensating control, but the exception authority and expiration date must be explicit.
Evaluation governance protects the evidence itself. Freeze model and system versions, preregister the protocol, separate test construction from release ownership when risk warrants it, track benchmark exposure, and retain prompts and outputs under appropriate access controls. Include ordinary use, foreseeable misuse, adversarial testing, subgroup analysis, and human-system interaction. Independent evaluation should be proportional to risk; it adds value only when the evaluator can inspect enough of the system and report unfavorable results without pressure.
A gate is a lifecycle control, not a one-time ceremony. Use staged deployment, exposure limits, canary groups, monitored rollback thresholds, and an expiry date for the evidence packet. Re-run affected tests after changes to weights, data, system prompts, tools, filters, or operating context. Diagnostics should show pass rates with intervals, failed scenarios, subgroup coverage, evaluator agreement, unresolved exceptions, and distance to each threshold. The final record should make the decision reproducible by someone who did not attend the release meeting.