Safety Agents for Autonomous Systems

ControlOps: Letting Machines Talk ControlOps: Letting Machines Talk | Stack Research Corporation The earlier posts in the “Let Machines Talk” series laid out four ideas: kill paths for stopping systems safely, blast radius containment for limiting damage, decision lineage for knowing why a system did what it did, and rollback for undoing it. Each post described the concept in isolation. ControlOps is what happens when you wire them together. It’s a set of agents that implement these ideas as composable operations — small, single-purpose programs that can run independently or be chained into pipelines. The Agents There are four, each handling one concern. Scope validator. Before any action executes, this agent checks the proposed action against governance requirements. It takes a description of what the system wants to do and returns a verdict: pass or fail. If it fails, it includes findings — which rules were violated and why. Every action earns its right to exist, or it doesn’t happen. Lineage recorder. After a decision is made, this agent structures the decision event into an append-only lineage record. It captures the input, the reasoning, and the output in a format that can be queried later. This is the decision lineage concept from the earlier post, turned into a concrete operation. Each record gets an integrity check so you can verify it hasn’t been tampered with. Blast radius assessor. Given a service’s permissions, dependencies, and resource limits, this agent estimates how much damage the service could cause if it failed. It returns a risk score, a maximum damage potential estimate, expected detection latency, containment time, and recommended controls. This is the “what’s the worst this can do?” question from the containing failure post, answered systematically instead of on a whiteboard. Kill path auditor. This agent audits a system’s shutdown capabilities against the four-level kill path spectrum — throttle, degrade, isolate, hard stop. It checks which levels are implemented, identifies gaps, evaluates escalation readiness, and recommends actions to close the gaps. Two Pipelines The agents are useful alone, but the real value is composition. ControlOps ships two pipelines that chain agents together. Governance Pipeline This is the pre-execution gate. Before a target agent runs, the pipeline: Runs the scope validator against the proposed action. If validation passes, executes the target agent. Records decision lineage for the entire sequence. Creates a checkpoint of the resulting state. If scope validation fails, the pipeline short-circuits. It still records lineage and creates a checkpoint — you want a record of what was rejected and why — but the target agent never runs. The action doesn’t happen. This is the “explicit scope” requirement from the original post, implemented as a pipeline stage rather than a hope. Resilience Pipeline This is the system health audit. It composes two agents: The blast radius assessor evaluates the system’s damage potential. The kill path auditor evaluates the system’s shutdown readiness. The pipeline combines both results into a single resilience verdict: adequate, partial, at-risk, or inadequate. A system with a high risk score but strong kill path coverage might land at “partial.” A system with a high risk score and no kill paths is “inadequate.” This is the blast radius design metric from the containing failure post — maximum damage rate times detection time times containment time — evaluated by agents instead of estimated by people during a design review. Why Agents These operations could be functions. They could be scripts. Making them agents means they can run in different modes — rule-based for speed and determinism, or LLM-backed for flexibility when the inputs are unstructured. A scope validation check against a well-defined policy can run as a pure rule engine. A scope validation check against a vague requirements document benefits from language model reasoning. The agent abstraction also makes composition straightforward. Each agent has a defined input schema and output schema. Pipelines are just agent sequences where one agent’s output feeds the next agent’s input. Adding a new concern — say, a cost estimator or a compliance checker — means writing a new agent and inserting it into the pipeline. The full source is at stack-research/agents under catalog/projects/control-ops . What This Doesn’t Do ControlOps handles governance, lineage, blast radius assessment, and kill path auditing. It doesn’t handle rollback execution. Rollback is a harder problem because it requires deep integration with the specific systems being rolled back — the database, the payment processor, the message queue. ControlOps gives you the lineage records and blast radius assessments that make rollback planning possible. The actual undo is still a problem to solve. It also doesn’t replace thinking about these concerns. Running a blast radius assessor against a service description is useful. Designing the service to have a small blast radius in the first place is more useful. The agents catch what you missed. They don’t substitute for getting it right. The earlier posts in this series argued that autonomous systems need explicit scope, verifiable lineage, containment boundaries, and kill paths. ControlOps is one way to implement those requirements as operational checks rather than architectural aspirations. The agents are small. The pipelines are simple. The value is in making these checks a routine part of how systems operate, not a one-time design exercise that fades from memory the week after the architecture review.