Governance, Policy, and Risk-Proportional Approval

3. Enduring Principle

Humans own accountability; policy bounds execution

Humans define intent, acceptable risk, and the rules under which agents may act. Agents may propose decisions, gather evidence, execute authorized work, and recommend an outcome. They do not become the owner of business, legal, security, or operational risk.

The governing chain is:

Policy is executable organizational intent. Approval is a durable decision required by policy. Neither replaces the other.

Keep the authority concepts separate

Capability is what a system or agent is technically able to do.

Permission is what an identity is allowed to invoke at a resource boundary.

Policy is the contextual rule that determines whether an action is allowed, denied, or requires a decision.

Authorization is the resolved grant to perform a bounded action under a specific contract and policy version.

Approval is an accountable decision that satisfies a named gate. It is not general permission for future work.

Acceptance is the judgment that an outcome satisfies its governed contract. It occurs after evidence exists and is distinct from approval to begin work.

Exception is a time-bound, scoped departure from policy accepted by the appropriate risk owner. It is not a permanent policy edit or a precedent.

Confusing these terms creates hidden authority. For example, approving a Plan does not accept a WorkOrder result, and possessing repository write permission does not authorize an agent to use it outside the WorkOrder.

Resolve authority before execution

Every material action should trace to an authorization envelope that freezes:

• the Mission, approved Plan version, WorkOrder, and responsible owner;
• actor and execution identity;
• repository, environment, resource, and path scope;
• permitted tools and operations;
• risk classification and applicable policy version;
• required approvals and their validity periods;
• budget, runtime, retry, and concurrency limits;
• validation and evidence requirements;
• recovery, cancellation, and escalation rules; and
• the exact facts used in the policy decision.

If the factory cannot compute this envelope, the safe answer is not “best effort.” Material execution should stop with a specific remediation.

Risk determines the control depth

A practical first model uses three bands:

The label should be policy-derived and explainable. A useful classifier considers:

risk = impact × likelihood × exposure × irreversibility × uncertainty

Detection strength and recovery quality reduce residual risk; they do not erase the underlying hazard. The system should retain the factors, not only the color.

Decision rights must be explicit

The following matrix is a defensible starting point. Organizations can change the titles, but they must preserve accountable ownership.

The factory should govern deployment but need not perform it. GitHub Actions, Jenkins, Argo CD, Spinnaker, Azure DevOps, or another delivery system may execute the release. The factory owns the policy decision, evidence contract, approval state, and lineage connecting the decision to the external execution.

Separate proposal, execution, validation, approval, and acceptance

Separation of duties reduces correlated error and prevents self-authorization. A robust flow distinguishes five responsibilities:

One person may hold several roles in a small company, but implementation and validation must remain technically distinct. Validation should run through a separate identity and execution path, use predefined criteria, generate its own evidence, and lack permission to modify the artifact under evaluation.

Escalate disagreement; do not vote it away

Validator disagreement increases governance. It never decreases it.

When credible validators conflict, the factory should create a Risk Review that contains the competing claims, methods, artifact identities, severity, freshness, independence, likely causes, and safe options. The next action may be targeted revalidation, corrective work, a domain-owner decision, or rejection. An unexplained retry is not resolution.

Review evidence, not agent activity

Approval fatigue appears when reviewers repeatedly receive low-information requests. The remedy is not merely fewer approvals; it is higher-quality decisions and better policy.

An evidence-centered review should show:

• the decision required and its accountable owner;
• the business intent and approved scope;
• why policy raised the gate;
• the exact artifact, commit, environment, and proposed action;
• criterion-level evidence, failures, conflicts, staleness, and waivers;
• material deviations and surprises;
• residual risk and rollback strategy;
• a recommendation and its uncertainty; and
• what resumes automatically after the decision.

Routine work that remains inside policy should proceed without repeatedly asking a human to reaffirm the same boundary. Escalate surprises: new risk, missing authority, conflicting evidence, exhausted recovery, or policy breach.

Exceptions are governed objects

An exception must identify the rule being bypassed, scope, owner, reason, expiry, affected artifact, compensating controls, and review requirement. It must be auditable and revocable. Expiry should return the system to the normal rule automatically.

Changing a label from “confidential” to “public,” marking a failed test as passed, or broadening an authorization envelope is not an exception mechanism. It is evidence or policy tampering.

Trust determines eligibility, never authority by itself

Operational autonomy and trust calibration are developed in Operational Autonomy and Trust Calibration. The critical governance rule is that Trust Score is a ceiling. It may reduce the maximum autonomy a factory can request, but it cannot override policy, create permission, or approve a decision.

The internal score may use 0–100 for trend computation while operators see stable bands:

Promotion requires sustained evidence and an explicit human decision. A defensible Level 2 to Level 3 default is at least 100 successful WorkOrders, 30 days of stable operation, at least 99 percent independent-validation success, zero critical security or policy violations, zero unauthorized actions, and human promotion approval.

Demotion can be automatic. Security or policy violations, unauthorized action, evidence tampering, fabricated results, high-risk validation failure, customer-impacting regression, repeated boundary violations, or suspected tool compromise should immediately lower authority or quarantine execution pending review. Older failures may lose scoring weight in a rolling window, but they never disappear from audit history.

Govern learning as a change to the factory

The factory may automatically collect outcomes, failures, metrics, and proposed improvements. Changes to prompts, policies, workflows, evaluation criteria, model routing, or authority alter system behavior and therefore require explicit human review and promotion. Continuous observation is compatible with governed learning. Unreviewed self-modification is not.

8. Notes and lessons learned

The most important lesson is that governance is not an approval screen. It is the architecture that turns organizational intent into bounded machine authority.

The failed golden-path run made this concrete. A Plan and WorkOrders existed, but execution correctly remained unavailable because the operational boundary was incomplete. The missing policy was a missing answer to “under what rules may this executor act?” The missing GitHub App was a missing answer to “with which identity and least-privilege authority may it cross the repository boundary?”

The Mission Control implementation also demonstrates why evidence-backed writing matters. Policy envelopes, approvals, Factory readiness, and scoped roles are real. A universal fail-closed policy service, canonical decision rights, Risk Review, Trust Score, and automatic quarantine are not yet proven. The architecture can be respected without converting its roadmap into a current-capability claim.

Questions to revisit after todo 024 and the next lab run:

• Does every executor and tool call use the same policy decision path?
• Can an agent identity satisfy any material human approval?
• What is the precedence rule when tenant, workspace, Factory, and WorkOrder policies disagree?
• Which changes invalidate an existing approval?
• Can late or duplicate events reopen authority after expiry or cancellation?
• Is the policy decision trace understandable to an operator without reading source code?

9. Interview and discussion questions

1. What is the difference between permission, policy, authorization, approval, acceptance, and exception?
2. Why should an AI Software Factory govern deployment even when an external CI/CD system performs it?
3. How would you design policy precedence across company, workspace, repository, Factory version, Mission, and WorkOrder scopes?
4. When should a system fail closed, and when is degraded advisory operation acceptable?
5. How do risk-proportional gates reduce approval fatigue without weakening accountability?
6. Why is validator majority voting unsafe?
7. How can one person in a startup hold multiple roles while preserving independent validation?
8. What evidence should be retained with a policy decision?
9. Why must Trust Score remain subordinate to policy?
10. Which events should cause immediate autonomy demotion or quarantine?
11. How would you explain Mission Control's current policy fallback to a CTO without overstating either the defect or the product capability?
12. Why did the golden-path run stop after Plan approval, and why was stopping the correct outcome?

Teach-back prompts

Developer: Explain how a WorkOrder becomes authorized without treating a GitHub permission as sufficient authority.

CTO: Defend the cost of policy evaluation, independent validation, and risk-proportional approval in terms of speed, quality, and organizational risk.

CEO or board: Explain why “trust the system, not the model” permits useful autonomy without claiming that AI is reliable by itself.