Keeping the human in the loop

At Deliverance AI, we've been thinking about what "human in the loop" actually means as AI agents take on more work inside enterprises.

There is a risk that people gradually lose the hard-earned skills and judgement they need to question the system. They may remain responsible for approving decisions while becoming less equipped to assess them.

A recent paper by researchers at Hugging Face and Data & Society sets out this possibility. As automation becomes more capable, human operators can become less prepared for when their judgement is needed most.

Passive monitoring can lead to approval fatigue, while domain knowledge fades through lack of use. A fluent answer becomes easier to accept than examine properly. Over time, human oversight can become a procedural step, with people approving outputs they have neither the time nor the confidence to challenge.

This is where "use it or lose it" applies to judgement. We need to keep the pitter-patter of grey matter going.

In practice, that could mean forming an independent view before seeking an AI recommendation, or periodically completing selected tasks without assistance. Teams should also challenge the evidence behind an output and practise taking control when an agent gets something wrong.

Those habits need support from the systems people use. Reviewers need access to the relevant evidence, clear authority to intervene and enough time to make a considered decision. Asking someone to approve an answer without those conditions offers little meaningful oversight.

As agentic AI takes on more operational work inside large organisations, governance must develop at the same pace as autonomy.

At Deliverance AI, we believe good governance must protect human capability alongside controlling what AI agents can do. The human in the loop has to remain an active participant, with the knowledge and confidence to question the system.