For the last few years the AI conversation inside companies has been about adoption. Which model, which assistant, which workflow, can we connect it to our own documents. Good questions, all of them. There is another one arriving fast: can you prove what your AI is doing, what it can reach, what risks it brings and who is accountable for it?
That question is not coming from a vendor. It is coming from regulation, from customers, and increasingly from your own security team.
AI moved from experiment to infrastructure
AI used to sit at the edge of the company. A few developers with API keys. Someone's internal chatbot. A Copilot licence that nobody tracked.
Now it is in development environments, support desks, security operations, HR systems, document processing and infrastructure management, and it is starting to run unsupervised in agent workflows.
The difference matters more than the hype around it suggests. An employee asking a chatbot to rewrite an email is a risk you can reason about in a sentence. An agent that can read your repositories, query production, open internal documents and call tools is a different animal. Identity matters. Permissions matter. Data boundaries, logging, the model version, the tools wired to it, all of it matters, and every one of those decisions has to be explainable months later to someone who was not in the room.
Securing AI stopped being about protecting an API key some time ago.
Europe is putting rules around that shift
The EU AI Act is now moving out of legislation and into application. Most of it became applicable on 2 August 2026. Prohibited practices and the AI literacy duty started earlier, general-purpose model obligations in August 2025, and the heavy high-risk requirements are phased in later: December 2027 for Annex III systems, August 2028 for certain AI built into regulated products. I went through that timeline in more detail in the pharma-focused piece.
This does not mean every company using an LLM is suddenly operating a high-risk system. The Act is deliberately risk-based. A spam filter is not treated like a system deciding who gets hired, who gets credit or who gets access to an essential service.
The direction is the point. AI has stopped being a procurement decision and started being a governance one.
The interesting part is not the fine, it is the engineering
Regulation gets discussed in terms of penalties, which is the least useful part of it. Read what the Act actually asks of high-risk systems and a familiar pattern shows up: risk management, technical documentation, logging and traceability, human oversight, quality management, accuracy, robustness, cybersecurity.
Three of those are worth reading properly if you build systems.
- Article 9 describes risk management as "a continuous iterative process planned and run throughout the entire lifecycle", with regular systematic review. Not a document produced two weeks before launch.
- Article 12 requires logging capability that supports traceability and post-market monitoring. Records you can go back to.
- Article 15 requires resilience against attempts to alter a system's use, outputs or performance, and names AI-specific threats: data poisoning, model poisoning, adversarial inputs designed to make the model get it wrong, and confidentiality attacks.
Those are legal requirements in a specific context, and I am not suggesting they apply to your internal chatbot. But strip the legal framing and they describe something security engineers have been saying for two decades. If a system matters, you need to know what it is doing, and you need evidence that your controls actually work.
The Commission is pushing the same way from the security side. Its July 2026 action plan on cybersecurity and AI puts Member States, ENISA and industry to work on security blueprints and testing for advanced AI in critical sectors.
AI makes the evidence problem worse
Ordinary software is already hard to govern. You have source code, dependencies, infrastructure, identities and data flows, and most organisations struggle to describe all five accurately.
An AI system adds a second layer on top of that one:
- foundation models from external providers,
- prompts that quietly shape behaviour,
- vector stores and knowledge sources,
- model and agent permissions,
- tool integrations,
- training or fine-tuning data,
- guardrails,
- model versions that change without anyone touching your application,
- and agents deciding for themselves which tool to call next.
So the question stops being "is the server secure" and becomes "what have we actually built here, what can each part of it do, and can we show that the controls around it hold". Plenty of organisations have a policy answer for that. Far fewer have an evidence answer, and the reason is usually the same: the evidence is scattered.
Compliance cannot live in a spreadsheet forever
This is the problem we keep running into with WardBee by ScanComb. The cloud evidence is in AWS. The workflow evidence is in GitHub or GitLab. The policies are in a document store, the tickets are somewhere else, the framework requirements are in a PDF, and when someone needs to demonstrate compliance, a human being manually assembles the story out of all of it.
That workflow barely coped when infrastructure changed a few times a quarter. It does not cope at all when developers, coding assistants and automated workflows are changing things continuously. I wrote about that specific collision in evidence for cloud compliance in the age of vibe coding.
WardBee's approach is to connect read-only technical evidence to compliance controls and keep confirmed evidence strictly separate from what is unknown, stale, conflicting or waiting on a human. AI is genuinely useful for explaining a finding or drafting remediation. It cannot turn missing evidence into a pass, and any tool that appears to do so is doing something you should look at closely.
The AI Act is one signal, not the whole story
Nobody is dealing with the AI Act in isolation. European companies already carry GDPR, NIS2, the Cyber Resilience Act, whatever their sector adds, plus ISO 27001 or SOC 2 because customers ask. What applies to you depends entirely on what you build and where you sell it.
Underneath, they rhyme: know what you operate, know the risks, protect it, record what happens, assign responsibility, keep the evidence.
That is not paperwork. That is an engineering discipline, and it belongs where the systems are: controls connected to the things they describe, evidence collected from the source, changes evaluated as they happen, and humans spending their time on the judgment calls that actually need judgment.
Security and compliance are collapsing into each other
There used to be a comfortable separation. Engineering built the system, security protected it, compliance documented it afterwards.
AI makes that border impossible to hold. An over-permissioned agent is an engineering problem, a security problem and a governance problem at the same time. So is a model version that changed under you, an undocumented data source, a missing audit log, or an AI system that quietly became business-critical while nobody was looking at it.
Which is why I do not think the AI Act produces another checklist. It is one more signal that companies need real visibility into what they are running.
The next AI question
I am still optimistic about this technology. Models get better, agents get more capable, software gets faster to build. That part looks inevitable.
But the companies that get AI into serious production will need something past model capability. They will need control, and not the kind that stops people using AI. The engineering kind: know what is running, know what it can do, know what protects it, and be able to prove it.
Experimentation was the first phase. Adoption was the second. Now AI becomes normal infrastructure, and normal infrastructure eventually gets asked a very ordinary question. Can you show me the evidence?