Loading
When Should an AI Agent Stop Running and Hand Decisions to a Person?
·7 min read
An AI agent rarely completes a task in a single step. It retrieves information ⇨ selects a tool ⇨ acts on a system ⇨ checks the result and ⇨ decides what to do next. As this sequence unfolds, the conditions under which it began can change. A source may contradict earlier information, a tool may return an unexpected result, or a later action may fall outside the authority granted at the start. If the agent continues without accounting for that change, an early error can shape everything that follows.
The system therefore needs a reliable way to interrupt execution. This is a question of how the agent is controlled while it works: how changes in its operating conditions are detected, what causes an intervention, and how the process can be paused without losing control of the work already completed.
The 2026 Singapore Consensus on Global AI Safety Research Priorities makes an important distinction between two principles: runtime assurance and interruptibility.
Runtime assurance concerns whether an agent continues to operate within the conditions under which it was validated, even as its inputs, accumulated state and environment change. Interruptibility gives human operators the ability to pause, redirect or stop its execution, and to reverse its effects where possible. One addresses whether the system is still operating as intended; the other provides a way to intervene when it is not.
The distinction has a practical consequence for system design. A shutdown mechanism cannot depend entirely on the agent recognising a problem and deciding to obey a stop instruction. The Consensus recommends that shutdown and override controls operate outside the agent's reasoning loop, through system-level privileges. It also calls for continuous monitoring and detection of changes in the agent's state independently of its own reporting.
This separation matters when an agent encounters manipulated information, follows an unexpected sequence of actions or enters a state it cannot reliably assess. The same system whose behaviour needs to be interrupted should not have sole control over whether the interruption takes place. The execution environment and orchestration layer need to retain that authority.
An agent may be validated for a particular set of permissions, data sources and operating conditions. During a longer task, those conditions can change. New information enters its context, external systems return different results, and actions accumulate in its memory. The agent may eventually be operating in circumstances quite different from those in which it was approved for deployment.
A retrieved document could contain instructions that conflict with the task's original purpose. Two connected systems might report different information about the same transaction. A tool might repeatedly fail, prompting the agent to try alternative routes that were never part of the intended process. Each development can affect what the agent does next.
Organisations need to define which changes require an intervention before deployment. These may include attempts to access restricted information, actions that exceed an approved spending or authority limit, conflicting information from trusted sources, repeated execution failures, or activity outside the expected pattern. A model's own assessment of its confidence can contribute to monitoring, but it cannot serve as the only safeguard. Some failures are precisely the ones the agent may not recognise.
A 2026 case study in the Harvard Data Science Review illustrates how this works in business operations. Its supply-chain management system uses multiple agents to monitor purchase orders, detect missing or mismatched information and communicate with internal teams, suppliers and carriers. Exceptions are recorded for managers to review, while one-off manual corrections remain subject to their approval. Routine monitoring and communication can proceed continuously, while decisions requiring further intervention remain under human control. AI Agents Are Transforming Decision Making: What Leaders Should Know describes the arrangement.
The important design choice is where the system draws its operational boundaries. An unexpected result does not necessarily require shutting down every agent involved in a task. It may be enough to prevent a particular update or correction from proceeding until the underlying information has been checked.
Interrupting an agent should not automatically mean terminating the entire process. The appropriate response depends on what has happened and how serious the anomaly is. A routine tool failure may call for a pause and reassessment. Suspected manipulation of the agent's context may require isolating its session. An attempt to carry out an action beyond its authority may require disabling the relevant permissions altogether.
The Singapore Consensus explicitly recommends graduated intervention, from throttling and pausing through isolation to full termination. It also identifies the ability to redirect an agent during execution without terminating the entire workflow as an important engineering requirement.
This distinction becomes particularly important once an agent has already acted. Stopping further execution does not reverse a record change, recall a message or undo an external transaction. Those effects need to be accounted for separately. The system should retain a reliable record of completed actions, identify any operations still pending, and provide tested rollback or remedial procedures where reversal is possible. The report emphasises that responsibility for intervention extends to remedying the agent's effects on its environment, not simply ending its execution.
A useful control mechanism therefore needs to account for the state of the workflow at the moment of interruption. It should be clear which actions have completed, which have been stopped and which systems may already have been affected. Otherwise, an operator may halt the agent while leaving behind partial changes that create further problems.
Once an agent has been paused, the next question is whether it can safely continue. The answer depends on why it stopped and whether the conditions that caused the interruption have been resolved.
A person reviewing the situation may authorise a specific action, provide missing information, change the agent's permissions or decide that the task should be abandoned. Any approval should apply to the action and conditions under review. A general instruction to continue can give an agent more latitude than the person intended, particularly when its original plan no longer reflects the current state of the task.
Before resuming, the system needs to establish that the relevant information is reliable, the required permissions remain valid and the next action falls within the revised scope. Where the circumstances have changed materially, the agent may need to reassess its plan rather than continue from an earlier point as though nothing happened.
Human oversight also needs to accommodate the volume of work agents can perform. The Singapore Consensus describes several complementary approaches, including approval checkpoints, monitoring during execution, batch reviews and AI-assisted oversight. The level of direct involvement can reflect the nature of the task and the agent's demonstrated reliability. An agent that consistently operates within its defined conditions may require fewer approvals for routine actions. Repeated exceptions, unexpected behaviour or frequent human overrides should prompt a review of its permissions and operating assumptions.
The EU AI Act makes interruptibility a specific requirement for high-risk AI systems within its scope. Under Article 14, people assigned to oversee these systems must be able to intervene in their operation or interrupt them through a stop button or equivalent procedure that brings the system to a safe state. The obligation applies to high-risk systems covered by the Act, rather than to every AI agent deployed by a business. What the EU AI Act actually regulates explains the scope of those requirements.
An AI agent should pause when its operating conditions have changed, its actions move beyond the authority it was given, or continuing could allow an error to produce further consequences. That pause must be enforceable independently of the agent, preserve an accurate account of what has already happened, and give the organisation control over what happens next. The ability to stop an agent is ultimately part of the system's operating design, and needs to remain effective even when the agent itself is no longer behaving as expected.