Research Journal6 min readchapter
Initial AI Software Factory Research Canon
This canon establishes the first body of sources for the guide. It is not a claim that every source is correct or equally authoritative. It identifies the materials that should be studied, compared, tested, and challenged.
Claim boundaryThis is curriculum guidance. It does not by itself prove a production implementation.
This canon establishes the first body of sources for the guide. It is not a claim that every source is correct or equally authoritative. It identifies the materials that should be studied, compared, tested, and challenged.
Primary sources take precedence over commentary. Vendor documentation is primary evidence for what a vendor specifies or claims. It is not independent proof that the capability works in every environment. Benchmarks, research, tests, and direct operation supply different forms of evidence.
Agent engineering and protocols
| Source | Initial use |
|---|---|
| Anthropic: Building Effective Agents | Distinguish workflows from agents and study orchestration patterns. |
| Anthropic: Trustworthy Agents in Practice | Study the agent loop, meaningful human control, transparency, privacy, and security as autonomy increases. |
| Anthropic: Effective Context Engineering for AI Agents | Study context selection, compaction, memory, and long-running agent behavior. |
| Model Context Protocol specification | Use the authoritative protocol for hosts, clients, servers, tools, resources, authorization, and versioning. |
| Agent Client Protocol | Study the editor-to-agent boundary, transport model, capability negotiation, and version-compatibility responsibilities. |
| AG-UI documentation | Study event-based agent-to-user-interface integration, streaming state, human interaction, and protocol interoperability. |
| Agent2Agent Protocol specification | Study agent discovery, tasks, messages, artifacts, streaming, and long-running agent-to-agent collaboration. |
| OpenAI Agents SDK | Study agent definitions, orchestration, guardrails, state, tracing, and evaluation. |
| OpenAI: A Practical Guide to Building Agents | Study agent selection, models, tools, instructions, orchestration, layered guardrails, evaluation baselines, and human intervention. |
| OpenAI Responses API | Study tool-using model execution and durable application integration. |
| OpenAI: Unrolling the Codex Agent Loop | Study the relationship among the model, harness, tools, context, and execution loop. |
| Claude Code: Run programmatically | Study headless coding-agent invocation, structured output, session continuity, and automation boundaries as a dated product case. |
| Claude Code hooks reference | Study lifecycle interception, policy checks, event capture, and the risks of shell-level automation as a dated product case. |
| OpenAI: Harness Engineering | Examine an agent-first engineering environment as an industry case study, separating reported experience from independently verified general law. |
| Google DeepMind research | Track primary agent, evaluation, multi-agent, and safety research. |
| Google DeepMind: Co-Scientist | Study specialization, debate, ranking, and human accountability in a multi-agent system outside software engineering. |
| Microsoft AutoGen | Study event-driven and conversational single- and multi-agent runtimes. |
| Microsoft Semantic Kernel Agent Framework | Study enterprise agent abstractions and orchestration patterns; record experimental status where applicable. |
| LangGraph | Study durable execution, persistence, human-in-the-loop control, and low-level agent orchestration. |
Software engineering platforms and runtimes
| Source | Initial use |
|---|---|
| GitHub Copilot concepts | Track the boundary among assistants, coding agents, enterprise controls, memory, and agent management. |
| GitHub Agentic Workflows | Study natural-language automation compiled into permissioned GitHub Actions workflows. |
| GitHub Spark | Study natural-language application creation and deployment as a comparison case, not as the definition of a factory. |
| GitHub Actions | Study repository-native automation, runners, environments, permissions, CI, and deployment controls. |
| GitHub merge queues | Study current-head validation, required checks, queue batching, and protected-branch admission. |
| GitHub stacked pull requests | Study dependent change review and the coordination costs of ordered pull-request chains. |
| Git submodules | Study commit-pinned repository composition and its checkout, update, publication, and ownership tradeoffs. |
| Git subtree | Study copied-history repository composition and contrast it with submodules and coordinated repositories. |
| Devfile schema | Study declarative development-environment components, commands, events, resources, and portability. |
| CodeRabbit pull-request review | Study automated review as a dated product case, including incremental review and human resolution boundaries. |
| Convex overview | Study the reactive database, transactional server functions, scheduling, and durable application state used by Mission Control. |
| Convex functions | Distinguish queries, mutations, actions, transactions, and external side effects. |
| Hono documentation | Study the web-standard orchestration boundary used by Mission Control for long-running runtime integration. |
Data, knowledge, retrieval, and semantics
| Source | Initial use |
|---|---|
| Retrieval-Augmented Generation paper | Study the original retrieval-plus-generation formulation and separate research results from production retrieval engineering. |
| Dense Passage Retrieval paper | Study learned dense retrieval and compare it with lexical and hybrid retrieval under domain-specific evaluation. |
| Robertson and Zaragoza: The Probabilistic Relevance Framework | Study BM25 and the assumptions behind lexical relevance scoring. |
| Cormack, Clarke, and Buettcher: Reciprocal Rank Fusion | Study a simple method for combining ranked retrieval results without treating fusion as proof of relevance. |
| W3C SKOS Reference | Study standards for controlled vocabularies, taxonomies, thesauri, labels, mappings, and concept schemes. |
| W3C PROV-O | Study a standard model for representing entity, activity, and agent provenance across transformed knowledge. |
Evaluation, quality, reliability, and systems
| Source | Initial use |
|---|---|
| SWE-bench paper | Study repository-level evaluation design and the limits of reducing software engineering to issue resolution. |
| SWE-bench repository | Inspect current harnesses, datasets, variants, and reproducibility mechanisms. |
| NIST Secure Software Development Framework | Study requirements, design review, verification, provenance, vulnerability response, and continuous improvement across the SDLC. |
| NIST SP 800-218A | Study the AI-specific community profile for securing generative-AI model development across the software lifecycle. |
| NIST AI Risk Management Framework Core | Study documented TEVV, independent assessment, production monitoring, and AI risk decisions. |
| SLSA specification | Study source and build provenance, hardened build levels, artifact verification, and supply-chain threat boundaries. |
| in-toto Attestation Framework | Study typed, verifiable claims bound to immutable software subjects by digest. |
| DSSE | Study a typed signing envelope that avoids application-level canonicalization. |
| SPDX 3.0 | Study the current ISO-standard family for software bill-of-material and supply-chain data. |
| CycloneDX 1.7 | Study the current CycloneDX BOM model, lifecycle phases, and attestation predicate. |
| Sigstore Cosign verification | Study identity-aware signature, attestation, timestamp, and transparency verification. |
| RFC 8785 JSON Canonicalization Scheme | Study deterministic JSON representation for repeatable hashing and signing, including its constraints. |
| OMG Structured Assurance Case Metamodel | Study formal relationships among claims, arguments, evidence, context, and counterclaims. |
| Anthropic: Demystifying Evals for AI Agents | Study tasks, trials, graders, assertions, transcripts, evaluation layers, and production feedback for agents. |
| OWASP Top 10 for LLM Applications | Study prompt injection, excessive agency, supply-chain, output-handling, and resource risks for AI-enabled systems. |
| OWASP Top 10 for Agentic Applications 2026 | Study risks and mitigations specific to autonomous, tool-using, multi-step agent systems and MCP integrations. |
| OpenTelemetry Signals | Study correlated traces, metrics, logs, and baggage for production evidence. |
| OpenTelemetry Semantic Conventions 1.43.0 | Study shared telemetry vocabulary. Pin versions: CI/CD is release-candidate and GenAI conventions are moving/developing, so do not make them the authoritative factory schema. |
| NASA Systems Engineering Handbook Appendix | Study clear, singular, traceable, verifiable requirements and verification/validation matrices. |
| NASA Product Realization | Study the distinction between objective verification and validation in the intended environment. |
| DORA software delivery performance metrics | Study delivery throughput and instability without substituting activity metrics for outcomes. |
| Google Site Reliability Engineering books | Study reliability, toil, service levels, incident response, monitoring, and production operations. |
| Designing Data-Intensive Applications | Develop durable reasoning about state, consistency, replication, streams, failure, and distributed-system tradeoffs. |
Flow, organizations, and economics
| Source | Initial use |
|---|---|
| The DevOps Handbook | Study flow, feedback, continuous learning, deployment systems, and organizational change. |
| Accelerate | Study evidence-backed software-delivery measures, causal reasoning, and organizational performance. |
| Team Topologies | Study cognitive load, team boundaries, interaction modes, platforms, and fast flow. |
| Toyota Production System | Study flow, quality at the source, just-in-time work, automation with a human touch, waste, and continuous improvement without forcing a literal manufacturing analogy onto software. |
Mission Control
Mission Control is the primary implementation case study. Study notes must reference the exact commit, relevant product decision or ADR, source paths, tests, observed browser or runtime behavior, and known gaps. Mission Control documentation is evidence about Mission Control. It is not the universal definition of an AI Software Factory.
Canon maintenance
Review framework, product, and protocol links at least quarterly. Pin specification versions when a chapter depends on exact behavior. Preserve the access date and distinguish historical documents from current guidance.
Add a source only when it improves the ability to design, build, operate, govern, evaluate, lead, teach, or defend an AI Software Factory. Popularity alone is not a reason for inclusion.
External review
Review this chapter.
Challenge a claim, boundary, missing failure mode, unclear term, or unsupported evidence statement.
- Claim
- Boundary
- Failure
- Evidence