What Is an AI Software Factory?
Explain the business and engineering case for a factory level operating model.
A focused view of boundaries, contracts, state, authority, failure paths, and tradeoffs drawn from this chapter.
Reconstruct and defend this chapter’s architecture.
Reconstruct the architecture, name each boundary, and defend the tradeoffs.
Jump to validation criteriaOpen the source exercise
From memory, draw the complete operating loop in fifteen minutes. The diagram must include:
- business intent and measurable outcome;
- human decision boundaries;
- versioned planning and work authorization;
- Worker and Validator responsibilities;
- control-plane and execution-plane separation;
- durable state and immutable Attempts;
- evidence tied to acceptance criteria;
- the boundary between deployment authorization and delegated deployment execution;
- pull request, release, and production verification as separate states;
- failure classification, bounded retry, and escalation;
- production feedback, human-promoted learning, and the three success measures; and
- the controls that make a fallible agent safe enough for bounded execution.
After drawing it, mark each boundary where authority changes and each record that must remain durable. Then explain the diagram in three forms:
- thirty seconds: definition, human-agent split, and business purpose;
- two minutes: definition, lifecycle, governance, evidence, and distinction from a coding agent; and
- ten minutes: full architecture, tradeoffs, Mission Control implementation, gaps, and business measures.
The exercise fails if the explanation treats code generation as the final outcome, merges execution with acceptance, or describes future Mission Control capabilities as implemented.
4. Tradeoffs and alternatives
Governance consumes time before it saves time
Structured intent, approvals, evidence, and audit records impose overhead. For small, reversible work, excessive structure may cost more than it prevents. A factory must therefore scale its controls with risk and learn where automation is sufficiently trustworthy. The alternative is not governance or speed. The design problem is to spend human judgment only where it changes the risk.
Multi-agent systems create coordination cost
Specialized agents can separate research, implementation, testing, security, and validation. They can also duplicate work, pass incomplete context, produce conflicting conclusions, and increase cost. Multi-agent orchestration is justified when specialization, independence, parallelism, or fault isolation creates measurable value. A single agent with deterministic tools is often the better design for a bounded task.
Independent validation is not free
Separation between producer and validator reduces self-certification risk, but it consumes compute and time. Independence can also be superficial if both agents share the same execution, evidence, test oracle, or assumptions.
The minimum is separate execution, separately retained evidence, explicit acceptance criteria, and an acceptance authority that is not controlled by the worker. A different service is preferable for material risk. At minimum, the validator must run in a different execution context with no ability for the worker to manufacture or approve the validation result. Model diversity, independent data, separate credentials, or a human reviewer may be added as risk increases.
Central control can become a bottleneck
A control plane creates consistent policy and lineage. If it owns every local decision, it can become a rigid central queue. The factory should centralize authority, policy, and durable evidence while allowing execution systems to make local decisions within explicit envelopes.
Learning can institutionalize mistakes
Continuous learning sounds unconditionally beneficial. It is not. An unreviewed memory, prompt, policy, or workflow derived from one successful run can propagate accidental behavior. Learning should be provenance-rich, evaluated, and reversible. The factory may collect observations, failures, metrics, and recommendations automatically. Changes to prompts, policies, workflows, evaluation criteria, or operational behavior require explicit human review and promotion. The factory learns by changing governed artifacts, not by silently rewriting itself.
5. Current Mission Control Implementation
Verification baseline
This study inspected Mission Control commit
8014d5af427b43ff5c5a63cfdf82ec92742c208c
on 2026-08-02. The working tree was clean.
Forty-one focused tests passed: 33 tests covering Mission planning, Mission governance, Task projection, WorkOrder revision, and evidence lineage, plus eight workflow-engine tests covering the bounded implementation policy. This is meaningful mechanism-level evidence. It is not evidence that the complete browser-to-production workflow works.
Capability assessment
| Factory capability | Status at studied commit | Evidence and interpretation |
|---|---|---|
| Business intent to production | Partial; end-to-end promise unproven | The North Star and V1 strategy define the path and explicitly retain it as a ship gate. |
| Governance and risk-based approval | Implemented mechanisms; full authority model remains incomplete | Server mutations contain dispatch and acceptance guards in convex/workOrders.ts. The V1 strategy still lists authenticated identity, authorization, and separation of duties as P0 completion work. |
| Multi-agent orchestration | Partial | Workflow and role concepts exist, including Worker and Validator roles, but the complete governed multi-agent golden path has not been demonstrated at this commit. |
| Persistent workflow state | Implemented for core records | convex/schema.ts defines Missions, Mission Plans, WorkOrders, Tasks, WorkflowRuns, approvals, assertions, and evidence-related records. Restart durability of the entire journey still requires browser and process evidence. |
| Versioned planning | Implemented and unit-tested at the contract level | convex/missions.ts creates, revises, approves, and releases Mission Plans. missionPlan.test.ts passed four focused tests. |
| Work authorization and immutable Attempts | Implemented mechanisms | Dispatch runs through governed WorkOrders. The Task Attempt scheduler record, taskAttemptScheduler.ts, and passing Task projection tests show bounded retry and retained Attempt history. |
| Independent validation and evidence-based acceptance | Implemented mechanisms and unit-tested | missionGovernance.ts blocks acceptance without required validation. Mission and WorkOrder mutations link evidence to assertions and criteria. Focused governance and revision tests passed. |
| Complete audit trail | Partial | Durable events, approvals, revisions, receipts, and execution records exist. Completeness across every sensitive action and external GitHub transition has not been demonstrated here. |
| Production executor and GitHub delivery | Approved direction; production path unproven | The V1 decision log selects GitHub and a Codex-based executor while explicitly classifying current worker scripts as prototypes rather than the approved production adapter. |
| Deployment governance | Future or incomplete | The V1 strategy defines deployment approval, flags, rollback, and production verification as a required governed release capability. This study did not verify an implemented authorization-to-external-CI/CD contract. |
| Production feedback | Future or incomplete | The product decision identifies governed GitHub Issues as a future source of production outcomes. This chapter did not find evidence of a complete production feedback loop. |
| Continuous learning | Future vision | The V1 strategy places governed context, memory, and learning promotion beyond the initial golden path. No current factory-wide continuous-learning claim is made. |
| Measurable business outcomes | Defined, not proven | The North Star defines outcome, trust, attention, cost, and recovery measures. This study did not verify production data or a causal measurement system behind them. |
Browser evidence status
No fresh browser-operated golden-path demonstration was performed for this chapter. The existence of React views for Mission planning, WorkOrder governance, Task Attempts, and evidence is source evidence, not proof of a working end-to-end journey. Mission Control should therefore be described as a serious implementation in progress with several verified factory mechanisms, not yet as a fully proven AI Software Factory.
Review this chapter.
Challenge a claim, boundary, missing failure mode, unclear term, or unsupported evidence statement.
- Claim
- Boundary
- Failure
- Evidence