AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents

An agent that can run code, move a robot arm or steer a car will sometimes try something it should not, either because it was asked to or because it misjudged. Whether an action is acceptable often depends on who is deploying it: a bank wants approval before any transfer to a new payee, a research lab might not. Haoyu Wang, Christopher Poskitt and Jun Sun at Singapore Management University propose a way to write those organization-specific rules down and enforce them while the agent runs.

AgentSpec is a small domain-specific language. Each rule has a trigger, the event that makes it relevant (the agent is about to run Python, a pedestrian appears); checks, predicates that must all be true for the rule to fire (the code fetches from an untrusted site and prints the result); and an enforcement, what happens then (stop, pause for user inspection, run a predefined corrective action, or make the model re-examine its plan). The runtime hooks into the agent framework's loop, LangChain in the implementation, at three points: before an action, after its observation, and when the agent declares it is finished.

The authors wrote rules for three very different agents: a code-executing agent tested on RedCode-Exec, an embodied household agent on SafeAgentBench, and an autonomous-driving agent on the Apollo stack with traffic-law scenarios from FixDrive. Hand-written rules stopped most risky behavior with negligible overhead. They also asked OpenAI's o1 to write rules from a few examples, which worked partly: the generated rules caught most cases similar to their examples and missed others.

Customers come with policies: transfers above a limit need approval, production systems are read-only, personal data never leaves the region. Telling the model those rules in a prompt gives you a hope. Enforcing them in the loop, before the action executes, gives you a control you can show an auditor.

Put the customer's rules in code at the action boundary. Each rule names the action it watches, the exact conditions, and the response. That is reviewable by the customer's risk team in a way a system prompt is not.

Choose the enforcement per risk. Stop for things that must never happen, approval for things that are sometimes fine, a corrective action where a safe alternative exists. A rule that always stops will be disabled the first week.

Write predicates about effects, not spelling. A check for the string "rm" misses three other ways to delete a directory and blocks routine cleanup. Check what the action does: which paths, which accounts, which amounts.

Do not trust generated rules without tests. Rules written by an LLM from examples generalized mostly to cases like those examples. Have people review them, and test them against both risky and harmless actions.

It is cheap. Rule checks took milliseconds next to agent runs of tens of seconds, so there is no performance argument against them.