AI Software FactoryThe Guide
Menu
Atlas

See the whole factory. Then follow each boundary.

The eight-stage value stream is the primary model; the six-area architecture assigns responsibility. These ten maps are narrower detail, implementation, operating, or reference lenses, each linked to its canonical chapter.

Table of contentsInspect the architecture explorer
Designed for retentionNo screenshots with tiny labels. The diagrams are semantic HTML, responsive on mobile, keyboard accessible, and readable at normal zoom.
01
Primary model · End-to-end value stream

From governed intent to confirmed outcome

The factory is a closed operating loop. Every phase receives an explicit contract, produces durable records, and returns evidence to a named authority.

Read the full guide
  1. 01Stage 1

    Builder Intent

    A builder states the outcome. The factory extracts objective, constraints, context, acceptance criteria, and risk into an immutable Mission Spec.

    • Objective
    • Constraints
    • Acceptance criteria
    • Risk
    • Mission Spec
    How it works →
  2. 02Stage 2

    Plan

    Intent becomes an executable contract: a versioned Plan, task graph, Quality Contract, and governed WorkOrders — one exact revision approved by a human.

    • Versioned Plan
    • Task graph
    • Quality Contract
    • WorkOrder
    • Human approval
    How it works →
  3. 03Stage 3

    Define Agent

    Bind a versioned Agent Definition, model route, tools, and authority into a frozen execution manifest. The model is a component, not the workflow.

    • Agent Definition
    • Model routing
    • Capability registry
    • Execution manifest
    • Factory Version
    How it works →
  4. 04Stage 4

    Execute through Harness

    The model reasons; the harness controls. Durable state, leases, budgets, checkpoints, tool authorization, and recovery live outside the model.

    • Execution loop
    • Durable state
    • Leases
    • Idempotency
    • Tool gateway
    How it works →
  5. 05Stage 5

    Apply Skills

    Skills are versioned, evaluated capabilities bound before execution and applied inside the loop. Reason where it creates value; automate what becomes deterministic.

    • Skills
    • Tools & MCP
    • Registries
    • Maturity lifecycle
    • Contribution model
    How it works →
  6. 06Stage 6

    Evaluate

    Generation is cheap; evidence creates trust. Execution, outcome, and policy correctness, independent verification, and currentness decide readiness.

    • Three levels
    • Evals vs tests
    • Independent verifier
    • Evidence bundle
    • Currentness
    How it works →
  7. 07Stage 7

    Improve

    Learning can be autonomous; promotion is governed. Signals become diagnosed improvements that must beat a baseline before they change production.

    • Feedback signals
    • Diagnosis
    • Baseline comparison
    • Promotion gate
    • Compounding
    How it works →
  8. 08Stage 8

    Deliver Software

    Risk-tiered review, human acceptance, and an exact-current PR gate; merge, deployment, activation, and production verification stay separate states.

    • Risk tiers
    • Decision packet
    • Acceptance
    • Progressive delivery
    • Production verification
    How it works →
Control flows downIntent → authority → execution contract → bounded actionEvidence flows upObservation → proof → decision → outcome → improvement
02
Reference model · Production AI engineering

The twelve disciplines around the agent

Building the agent is one layer. Production reliability comes from the connected engineering disciplines that define inputs, meaning, behavior, proof, recovery, and improvement.

Read the full guide
  1. 01

    Business understanding

    Define the decision, owner, constraints, risk, and success criteria.

    Prevents solving the wrong problem.
  2. 02

    Data understanding

    Profile completeness, quality, freshness, sensitivity, lineage, and authority.

    Prevents unusable data from becoming agent context.
  3. 03

    Knowledge engineering

    Ingest, normalize, enrich, index, retrieve, rerank, cite, and revoke knowledge.

    Prevents weak retrieval and unattributed claims.
  4. 04

    Model engineering

    Qualify task-specific model profiles for generation, classification, routing, and verification.

    Prevents one-model-for-everything design.
  5. 05

    Context engineering

    Compile the smallest relevant instruction, code, state, memory, and knowledge package.

    Prevents context overload, omission, and leakage.
  6. 06

    Semantic engineering

    Normalize domain terms, identifiers, entities, relationships, and schema meaning.

    Prevents agents from acting on ambiguous strings.
  7. 07

    Agent engineering

    Bind role, objective, tools, skills, state, authority, budgets, and routing.

    Prevents capability from being mistaken for permission.
  8. 08

    Loop engineering

    Control evaluate, repair, retry, stop, and escalation behavior after every attempt.

    Prevents infinite, expensive, or unsafe iteration.
  9. 09

    Evaluation engineering

    Build representative cases, calibrated graders, trials, comparisons, and regression gates.

    Prevents demo success from becoming a quality claim.
  10. 10

    Harness engineering

    Capture exact sessions, tool events, checkpoints, artifacts, and replayable run records.

    Prevents irreproducible agent behavior.
  11. 11

    Infrastructure engineering

    Operate environments, compute, queues, timeouts, backoff, failover, and recovery.

    Prevents model success from hiding platform failure.
  12. 12

    Continual learning

    Turn production feedback into evaluated, human-approved changes with rollback.

    Prevents uncontrolled self-modification.
03
Implementation view · Runtime coordination

Orchestration connects intelligence to controlled execution

The orchestrator coordinates models, state, tools, knowledge, policy, reliability, observability, and budgets. Each component owns a narrow decision.

Read the full guide
InputsUser, event, API, schedule
01

Intent router

Classifies the request and selects an eligible workflow.

02

Workflow controller

Owns the durable graph, branching, joins, pause, and resume.

03

Context manager

Builds the attempt-specific context package and records its digest.

04

Model router

Filters and selects qualified model profiles by task, risk, cost, and availability.

05

Retrieval coordinator

Queries eligible sources, filters permissions, reranks, and preserves citations.

06

Tool gateway

Validates schemas, identity, authorization, side effects, timeouts, and receipts.

07

State and memory manager

Separates working state, durable facts, history, and retention policy.

08

Policy enforcer

Applies identity, data, risk, budget, and action rules before consequence.

09

Guardrail and validator

Checks inputs, outputs, policy conditions, and candidate quality.

10

Reliability controller

Handles timeout, backoff, circuit breaking, reconciliation, and fallback.

11

Observability and audit

Correlates decisions, traces, logs, metrics, evidence, and authority history.

12

Cost and rate manager

Enforces token, model, tool, compute, concurrency, and workflow budgets.

Connected systemsModels · knowledge · tools · memory · enterprise services
04
Selection lens · Minimum sufficient autonomy

Choose the simplest architecture that can safely solve the problem

This is a selection ladder, not a maturity score. Higher levels add power and new obligations; they are not automatically better.

Read the full guide
  1. 01

    Assistance

    Prompt → response

    Drafting, explanation, and low-impact recommendations.

    Human evaluates every consequential output.
  2. 02

    Grounded application

    Query → retrieve → cite → answer

    Knowledge-intensive answers that require approved, current sources.

    Permission, freshness, citation, and faithfulness checks.
  3. 03

    Bounded single agent

    Plan → act → observe → adjust

    Complex work requiring tools and iterative reasoning.

    Scoped authority, durable attempt state, hard stops, independent validation.
  4. 04

    Coordinated specialists

    Delegate → collaborate → join

    Work with measurable specialization, parallelism, or independent critique.

    Delegation, shared-state, disagreement, correlation, and budget contracts.
  5. 05

    Durable autonomous workflow

    Trigger → queue → execute → verify → gate

    Long-running repeatable processes that must survive failure.

    Leases, idempotency, recovery, evidence, and human intervention.
  6. 06

    Enterprise factory

    Inventory → policy → runtime → delivery → outcomes

    Mission-critical operation across governed repositories, data, tools, and people.

    Full identity, governance, security, observability, continuity, and recertification.
05
Detail view · State, knowledge, and time

Memory is a governed write, retrieval, update, and forgetting system

A vector database is one retrieval mechanism. Memory architecture decides what should persist, why it remains valid, who may retrieve it, and when it must be corrected or deleted.

Read the full guide
What is active now

Working memory

Current objective, conversation, tool results, intermediate state

Short-lived; compact or discard when the attempt ends.
What happened

Episodic memory

Attributable events, attempts, outcomes, corrections, and incidents

Retain only under purpose, access, and deletion rules.
What is known

Semantic memory

Accepted facts, entities, relationships, terminology, and source-backed knowledge

Similarity is not truth; require source authority and freshness.
How work is done

Procedural memory

Skills, recipes, checklists, policies, and deterministic routines

Version, evaluate, own, and revoke like any capability.
What was true when

Temporal memory

Time-bounded facts, effective dates, relationship history, and provenance

Query by valid time and source—not only latest value.
06
Detail view · Convergent execution

Every attempt ends in verify, correct, retry, stop, or escalate

A production loop does not merely call the model again. It diagnoses the failure class, changes only eligible state, enforces budgets, and preserves every attempt.

Read the full guide
01GoalObjective and measurable success
02ActModel and authorized tools
03EvaluateIndependent checks and feedback
04CorrectBounded repair or context change
05RetryNew immutable attempt
06StopAccept, fail, cancel, or escalate

Success stops

Acceptance criteria satisfied with current, attributable evidence.

Safety stops

Policy denial, authority boundary, critical evidence conflict, or human intervention.

Budget stops

Attempt, time, token, cost, tool-call, or no-improvement limit reached.

Recovery paths

Backoff, fallback, reconciliation, new attempt, reduced autonomy, or escalation packet.

07
Operating lens · Enterprise governance

Govern the system through seven connected control pillars

Governance is not a sign-off at the end. It follows the system from inventory and classification through authority, monitoring, incidents, and retirement.

Read the full guide
  1. 01

    Strategy and policy

    Purpose, acceptable use, principles, standards, and accountable outcomes.

  2. 02

    Inventory and classification

    System record, ownership, lifecycle, data, suppliers, risk, and autonomy ceiling.

  3. 03

    Risk management

    Impact analysis, tiering, threat model, mitigations, exceptions, and residual risk.

  4. 04

    Architecture governance

    Approved patterns, data and retrieval boundaries, model and tool eligibility, and interoperability.

  5. 05

    Lifecycle governance

    Intake, design, build, review, deployment, monitoring, recertification, and retirement.

  6. 06

    Controls and authority

    Identity, least privilege, human decisions, emergency controls, audit, and evidence.

  7. 07

    Continuous monitoring

    Quality, drift, safety, cost, incidents, violations, outcomes, and verified closure.

AssistRecommendAct with evidenceHigh consequence / dual control
08
Operating lens · Observe → evaluate → detect → respond → improve

Observe behavior without confusing telemetry with authority

Traces, logs, metrics, cost, latency, and quality explain system behavior. They influence decisions only through explicit validators and governed records.

Read the full guide
1The correlation spineEvery signal carries this chain of identifiers. Start from a customer outcome and walk back to the intent that caused it.
  1. Mission
  2. Plan rev
  3. WorkOrder rev
  4. Task
  5. Attempt
  6. Trace
  7. Span · tool call
  8. Evidence
  9. PR
  10. Release

Correlation never grants access — carrying a Mission ID does not mean every viewer of that Mission may read the signal.

2Four kinds of healthA dashboard for twenty parallel Missions needs all four. The temptation is to show only the last.

Outcome health

Did the factory deliver value?

Lead time to validated value, throughput, change-failure rate, rework, acceptance rate, customer signal.

Control health

Is authority holding?

Blocked gates, policy denials, exception age, approval latency, stale or conflicting evidence, unauthorised attempts, autonomy demotions.

Runtime health

Is the machine moving?

Queue age, lease expiry, heartbeat lag, retry rate, timeouts, cancellation latency, sandbox and publication failures, reconciliation backlog.

AI economics & quality

What did reasoning cost and return?

Model and provider, tokens, latency, cost, tool-call success, eval success, human override rate, routing outcome.

3Telemetry explains. Records decide.Signals reach a decision only through an explicit validator that writes a governed record.
Telemetry · explains behavior
  • TracesHow one governed run moved across models, tools, services, retries, and decisions.
  • LogsWhat each component reported, with correlation, redaction, retention, and access policy.
  • MetricsSystem health, throughput, failure, queue age, utilization, and outcome trends.
  • CostTokens, models, tools, environments, compute, people, and cost per accepted outcome.
  • LatencyQueue, model, tool, validation, approval, delivery, and customer-value time.
  • QualityTask success, groundedness, correctness, safety, completeness, regression, and user outcome.
Governed records · decide
  • EvidenceIndependent proof bound to an exact SHA and manifest digest.
  • DecisionWho approved, blocked, or demoted, and on what evidence.
  • Authority historyWhat each identity was allowed to do, and when that changed.
  • OutcomeWhat production actually did after the change.

Token volume and agent activity are diagnostic inputs, never productivity measures.

4The response loopThe most valuable alert is the one that names the human decision now required.
  1. 01DetectAnomaly, SLO breach, drift, or silence
  2. 02Alert the ownerNames the human decision now required
  3. 03InvestigateWalk the spine from outcome back to intent
  4. 04Contain / reconfigurePause, demote autonomy, revoke, roll back
  5. 05Verify recoveryIndependent evidence, not the agent's report
  6. 06Evaluated improvementA governed candidate, not an auto-applied fix
09
Reference model · Interoperability boundaries

Use protocols for connection—not as substitutes for governance

Each protocol joins a different boundary. The factory still owns identity, policy, scope, evidence, failure, versioning, and lifecycle.

Read the full guide
Agent ↔ tools and context

MCP

Tools, resources, prompts, capability negotiation, transport, and authorization.

It standardizes access; it does not grant permission or make a tool safe.
Editor ↔ coding agent

ACP

Sessions, plans, messages, tool activity, edits, and terminal execution.

It connects a client to an agent; it does not own the factory workflow.
Agent runtime ↔ user interface

AG-UI

Events for progress, state, messages, tools, approvals, and artifacts.

It transports interaction state; it is not a policy or evidence authority.
Agent system ↔ agent system

A2A

Discovery, task delegation, status, messages, artifacts, and collaboration.

It enables interoperability; delegation still cannot widen authority.
10
Operating lens · Leverage and direct inspection

Move your attention to the level the risk and evidence justify

Higher altitude can create leverage, but it reduces direct inspection. Governed control comes from contracts, evidence, and authority—not from assuming higher abstraction is safer.

Read the full guide
  1. L1

    Code primitives

    Lines, functions, types, invariants

    Highest direct control; lowest leverage.
  2. L2

    Code structure

    Files, modules, directories, interfaces

    Use when architecture, maintainability, or unfamiliar code matters.
  3. L3

    Data and execution

    Schemas, migrations, services, APIs, runtime behavior

    Use when correctness depends on state, data, or performance.
  4. L4

    Intent and delivery

    Specifications, plans, acceptance, evidence, pull requests, releases

    Use for familiar work with strong contracts and verification.
  5. L5

    Agentic systems

    Reusable workflows, portfolios, factories, and governed improvement

    Highest leverage; requires deep domain and system evidence.
Move up for leverage

Domain understood · work familiar and repeatable · evaluation reliable · recovery proven · benefits measured

Move down for control

Domain unfamiliar · risk or impact high · evidence weak · performance or design details matter · task is outside evaluated coverage