Control Tower Monitoring, Detection, and Response

1. The problem

Individual dashboards show calls, latency, cost, test results, policy events, or incidents. Operators need to know which governed system, release, autonomy grant, owner, and outcome are affected; what response is active; when it is due; and whether recovery has been independently verified. Without that spine, signals become unactionable noise and changes occur outside governance.

2. Enduring Principle

Operate through governed subjects and explicit response state

The control tower reads the system inventory and links telemetry and evidence to exact subjects. It does not become the source of truth for policy, workflow, incidents, or capability versions. Actions invoke the same authorized control APIs used elsewhere; UI and API behavior remain equivalent.

3. Response lifecycle

4. Control-tower subject model

Every view starts with a FactorySystemRecord and links current lifecycle, risk, autonomy ceiling, owners, releases, workflows, capabilities, models, tools, data, policy decisions, denials, exceptions, evidence freshness, dependencies, incidents, cost, performance, and outcomes. The current response records owner, severity, state, deadline, action, acknowledgement, verification, and escalation.

5. Signal and drift catalog

Drift can be data, semantic, context, model, prompt, tool, evaluator, workflow, policy, cost, reliability, or business-outcome drift. A statistical change is not automatically harmful; a policy violation may be critical without statistical significance.

6. Baselines, thresholds, and alert quality

Baselines are versioned by workflow, risk, repository class, tenant, model profile, and time window. Thresholds include absolute policy limits, rate/ratio changes, percentile shifts, budget burn, evidence expiry, and multisignal conditions. Each detection rule names owner, severity, window, minimum sample, uncertainty, false-positive disposition, deduplication key, suppression and maintenance rules, retention, privacy, and runbook.

Suppression never hides security incidents or control failures without a recorded exception. Repeated false positives produce a reviewed rule-change proposal; operators do not disable protection informally.

7. Response actions and authority

• Continue with observation: variation is explained and within policy.
• Pause: hold new steps at a safe checkpoint while preserving state.
• Cancel: end work and reconcile partial effects.
• Retry: repeat only under the operation's idempotency contract.
• Fallback: select a prequalified alternative with explicit changed limits.
• Reconfigure: follow change control; never silently mutate live policy or prompts from an anomaly.
• Rollback: restore a known version and verify data/outcomes.
• Quarantine: block selection and isolate the affected subject.
• Retire: remove authority and traffic, retain evidence, and delete by policy.

Each action displays requested effect, subject, authority, risk, evidence, deadline, expected acknowledgement, recovery implication, and alternate action.

8. Detection-to-closure contract

finding:
  id: finding-811
  subject: factory-system:payments-delivery@7
  rule: behavior-drift/tool-sequence@3
  baseline: baseline:bounded-change@12
  observed_window: 2026-08-30T17:00:00Z/2026-08-30T18:00:00Z
  evidence_refs: [trace-query:91, evaluation:44]
  severity: high
  owner: role:runtime-oncall
  deadline: 2026-08-30T18:15:00Z
  response: quarantine-capability-version
  control_ref: control-command:204
  state: verifying
  recovery_evidence: evaluation:49
  residual_risk: "Affected prior releases under review"

A finding progresses through new, triaged, responding, contained, recovering, verifying, closed, or escalated. Closure records detection quality, response, affected scope, verified recovery, residual risk, notifications, postmortem, and improvement disposition.

9. Failure modes

10. Accessibility and operator experience

Color never carries state alone. Tables, labels, timestamps, owner, severity, and next action provide a complete text equivalent. Keyboard users can select subjects, inspect evidence, and invoke authorized controls. Confirmations state the effect and recovery implication. Loading, empty, stale, permission-denied, partial-data, success, failure, and unknown states are explicit.

11. Tradeoffs and nonclaims

A unified view improves coordination but can become a dangerous administrative super-console. Keep it a least-privilege projection with narrow control APIs, step-up authorization, dual control where required, and full audit. This review-ready design does not prove detection quality, response time, closure effectiveness, or production accessibility.

12. Review exercise

Inject a quality regression, cost spike, stale evidence, and compromised tool version into one synthetic system. Demonstrate deduplication, triage, containment, owner escalation, rollback or quarantine, independent recovery verification, postmortem, and a change-controlled improvement proposal.