Control and governance
What the system is allowed to do — and what it is not
A reliable system is not born from what it is allowed to do, but from what it is forbidden to do.
Types of activity and level of automation
| Type of activity | What the system does | Human control |
|---|---|---|
| Low riskclassifying an incoming request, looking up information, standard replies | Acts on its own | Spot checks |
| Medium riskpreparing a quote, handling a complex request | Prepares and proposes | Approval required |
| High riskcontracts, significant payments, legal or people decisions | Analyses and assists | A person decides, always |
Never automatic, for any client, in any configuration: signing contracts, significant payments, legal decisions, decisions about people — hiring, dismissal, evaluation.
Casual use against professional work
| Casual use of AI | How we work | |
|---|---|---|
| Scope | A general-purpose assistant, the same for everyone | Agents aimed at one service and one specific process |
| Context | Does not know the business | Works on the company’s documents, method and language |
| Result | “Seems useful”, nothing measured | A process that used to cost hours closes in minutes — and the system measures what it does |
| Method | Changes on its own; invents when data is missing | Tells what the method provides from what is elaboration, cites sources, states the gap — and never changes the method by itself |
| Validation | No gate: the output goes out | Blocking human validation on anything that leaves |
| Control | Nobody knows what the agent does, who authorised it, what it can reach | Inventory of agents and their permissions, role-based permissions, traces out of the agent’s reach |
| Robustness | The control breaks at the second attempt | Controls hold against repeated attempts |
| Secrets | Keys end up in the agent’s context | The agent sees a reference, not the secret |
| Data | Leaves, and nobody knows where | Where the material lives is decided with the client |
| Incident | Discovered after it has happened | Who detects, who decides, who notifies and in how long — written in advance |
The difference is not the quality of the output. It is governability.
Security and control
No privileges by default, minimum access, full traceability.
Permissions of a role
Example: customer support
May
- read requests
- look up orders
- look up products
- prepare replies
- send standard replies
May not
- change prices
- delete customers
- make payments
- see salaries
- export the archive
Operating limits
| Refund under € 50 | the system may act on its own |
| € 50 – 500 | the system proposes, a manager approves |
| Over € 500 | the system may not act — escalated to a person |
Full trace
Every step is recorded: when the request arrived, how it was classified, which data was read, what was proposed, who approved, when the reply went out.
14:32:11 request received 14:32:12 classified as “support” 14:32:13 records consulted 14:32:14 order 8742 consulted 14:32:16 reply prepared 14:32:18 approved by the manager 14:32:22 reply sent
And not only what the system did, but why, with which data and under which authorisation. That is the difference between a system you can govern and one you have to trust.
Four questions we ask before we touch anything
- Where do the traces live — out of the agent’s reach, or inside the same system that can rewrite them?
- Do the controls hold against repeated attempts, or does a second try get through?
- Who holds the stop, and how long did it actually take the last time it was used?
- Does the agent see the secret, or only a reference to it? And who granted that permission?