There's a standard answer now to the question of how to keep company data out of ChatGPT. It goes like this: AI domains on the blocklist, SASE rule enforced, tenant restrictions on Copilot, enterprise headers required, train the staff. At the end of the list there's usually a quote for the matching endpoint suite.
None of that is technically wrong. I'd put most of those controls in place myself. As an answer to the problem it still falls short — for a reason familiar to anyone who has ever enforced a ban inside a company.
That's the part missing from the blocklist articles. Shadow AI doesn't come from carelessness. It comes from someone needing a proposal finished by 4:40 pm, with the tool sitting right there.
Take that away without putting something equivalent in its place and nothing is resolved. It moves. The document goes out via the personal phone, the home laptop, someone else's account — precisely where no logging, no data protection and no oversight reaches.
The core of it: after the block, the risk hasn't shrunk. It has become invisible. And invisible risks are the expensive kind — they surface only when someone outside the company points at them.
A ban without an alternative isn't control. It's giving up control while hoping nobody notices.
The answer that holds has three parts. The middle one is usually the one missing.
| Layer | Job | What happens without it |
|---|---|---|
| Block | Close public endpoints to company data | Data leaves in the open — the hygiene layer, and the endpoint crowd is right about it |
| Provide | Internal AI good enough for daily work | The block becomes a detour: shadow AI on personal devices |
| Govern | Who may do what, with which data — and how you prove it | The next audit finds an operation with no evidence chain |
Block: yes, still. Public endpoints have no business handling company data.
Provide: an internal AI good enough that nobody reaches for a private account. Not a fig-leaf chatbot that gives up after three questions — a model that actually does the work. This layer decides whether the first one holds.
Govern: the EU AI Act and ISO 42001 aren't a distant prospect any more, and NIS2 already asks for the evidence chain. Build the second layer without the third and you've handed yourself your next audit problem.
This is where sales conversations usually turn vague. So, concretely, from running it:
Open models have been good enough for everyday work for about a year now — summarising, drafting, code, analysis. Not for everything, but for most of it. The hardware is no longer a data-centre question: one machine in your own rack carries a production model for a team without a single token leaving the building.
In front of it belongs a router that keeps models interchangeable, so you're not back at the start in eighteen months. Behind it, logging, so the third layer has something to prove with. And access that feels like what people already know — otherwise the phone wins again.
I run this myself, not as a reference architecture on a slide. Which is also why I know where it gets uncomfortable: swapping models, who brings the box back up at night, and giving an honest account of what such a model cannot do. Anyone who doesn't tell you that hasn't operated it.
The argument isn't cloud versus on-premises. It's keeping control of your data without taking away the tool people use to do their jobs.
Blocking alone doesn't get you there. It just produces a company where everyone pretends they don't use AI.
The first layer can be read off a firewall rule. The third can't. The NIS2 check and the EU AI Act check show in a few minutes which of the three layers you can actually evidence — and which one you're currently just assuming.