Credal.ai actually does this for Enterprise usecases today (disclaimer: i am the founder). We use mostly the MCP framework, but you can set rules for human in the loop (HITL) based on the arguments for a tool call. (e.g. sending an email to yourself or your team is ok without HITL, but if anyone else is in to, cc or bcc then HITL is needed)
Comments
Credal.ai actually does this for Enterprise usecases today (disclaimer: i am the founder). We use mostly the MCP framework, but you can set rules for human in the loop (HITL) based on the arguments for a tool call. (e.g. sending an email to yourself or your team is ok without HITL, but if anyone else is in to, cc or bcc then HITL is needed)