Why we're looking at this
“Human in the loop” is the easy default, but it doesn't scale — if every action needs approval, the agent isn't saving anyone time. The interesting design problem is deciding exactly which actions can run autonomously and which need a human, and building that as an explicit, inspectable policy rather than scattered if-statements.
What we're seeing
- The clearest dividing line we've found is reversibility — actions that are cheap to undo (drafting a message, running a read-only query) are good candidates for full autonomy; actions that aren't (sending an email, writing to a production database) need an approval step or a strict allowlist.
- Policy needs to be enforced at the harness level, not the prompt level — asking a model nicely not to do something is not a safety control.
- Audit logging every tool call an agent makes, whether or not it required approval, has repeatedly been the difference between a fast root-cause diagnosis and a guessing game when something goes wrong.
Open questions we're still chasing
We're still working out how much policy logic should live in a shared, reusable layer versus how much is inherently domain-specific per client — early signs suggest the reversibility-based classification generalizes well, but the specific thresholds don't.