Workflow and Event Contracts, Schema Evolution, and Factory Storage
Define the durable contracts that connect triggers, workflow state, asynchronous events, artifacts, evidence, and retrieval stores.
A rapid review of the chapter’s existing Quick Read, principles, definitions, lessons, and review material.
The chapter in one pass.
- Purpose: Define the durable contracts that connect triggers, workflow state, asynchronous events, artifacts, evidence, and retrieval stores.
- Best for: Platform, data, runtime, and integration architects.
- Prerequisites: Runtime Orchestration and State Machines.
- Reading time: 15 minutes.
- You will learn: How to version workflows and events, evolve schemas, compensate partial effects, and choose authoritative versus derived storage.
- Keep three ideas: events are observations, not automatic authority; schema evolution is a rollout; and each store needs an explicit truth boundary.
Reconstruct and defend this chapter’s architecture.
Reconstruct the architecture, name each boundary, and defend the tradeoffs.
Open the source exercise
Connect a repository webhook, worker queue, artifact store, deployment provider, evidence store, search index, and analytics warehouse. Add duplicate and out-of-order events, a schema upgrade, and an ambiguous publication. Mark truth and reconciliation boundaries.
3. Enduring Principle
Version the workflow definition
A workflow contract or DSL declares nodes, inputs, outputs, dependencies, triggers, timeouts, retries, budgets, cancellation, compensation, human gates, completion states, and required evidence. Running WorkOrders remain bound to their approved version unless an authorized migration creates a new decision and replay-safe transition.
Treat triggers as intake, not authority
Schedules, webhooks, messages, API calls, and repository events create or update proposed work through authenticated, idempotent intake. Admission verifies current owner, scope, policy, risk, readiness, and budget before execution.
Use typed event envelopes
Every event includes canonical type and version, event and correlation identity, producer identity, tenant, subject, causation, occurred and received times, payload schema, integrity, and trace context. Consumers deduplicate and tolerate defined ordering. Unknown incompatible versions fail visibly.
Evolve schemas through compatibility windows
Additive changes usually precede consumer migration; semantic changes require explicit new versions. Producers and consumers advertise supported ranges. Backfill, dual-read or dual-write, validation, cutover, and contraction are monitored states rather than one deployment step.
Compensate partial effects explicitly
Distributed workflows cannot assume global rollback. Sagas record completed effects and approved compensations. Compensation is a new action with its own authority and failure path. Reconciliation handles ambiguous effects whose result is unknown.
Assign each store a truth boundary
| Store | Appropriate content | Important limit |
|---|---|---|
| Transactional control store | Authority, state, policy decisions, leases, approvals | Avoid large unbounded artifacts |
| Event log or message broker | Durable facts and asynchronous delivery | Delivery does not equal acceptance |
| Object or artifact store | Logs, diffs, test output, packages, evidence blobs | Metadata and digest must remain authoritative elsewhere |
| Search index | Discoverable projections | Derived, stale, rebuildable |
| Vector database | Semantic retrieval candidates | Similarity is not authority or evidence |
| Analytics warehouse | Aggregates and trends | Delayed projections cannot drive immediate authority blindly |
8. Notes and lessons learned
An event is evidence that a producer reported something. The control plane still decides whether that report is authentic, current, relevant, and sufficient for an authoritative transition.
9. Interview and discussion questions
- How do you migrate an active workflow version?
- Why is exactly-once delivery not the same as exactly-once outcome?
- What belongs in the event envelope?
- When should a vector database be rebuilt?
- How does compensation differ from rollback?
Review this chapter.
Challenge a claim, boundary, missing failure mode, unclear term, or unsupported evidence statement.
- Claim
- Boundary
- Failure
- Evidence