I have watched a language model explain a regulatory requirement in fluent, confident prose that a specialist would have signed off on, right up to the sentence where it cited a transition period that had expired two years earlier. Nothing about the answer looked wrong. That is the whole problem.
This is not an argument against using AI in regulated work. I use it daily and it has taken hours out of tasks that used to eat whole afternoons. It is an argument about one specific design decision that I see teams get wrong over and over: letting the model answer from memory instead of from the document.
A regulation is not a topic. It is a text, with a date, a scope and a set of exceptions, and every one of those can change without the wording of the summary changing at all.
What "from memory" actually means
When you ask a general model about a directive, a guidance document or a national implementing act, it answers from a compressed impression of everything it read during training. That impression is usually right about the shape of the rule. It is often wrong about the details that decide a case: which annex applies, whether a derogation still exists, what the notified body actually expects, which version of the guidance is current in which market.
Regulatory affairs is almost entirely made of those details. A specialist rarely gets asked "what does this regulation say?" They get asked "does this apply to us, in this market, for this product class, as of this quarter, and what do we need to show?" Every clause of that question depends on a source, and the source is a document with a number and a date.
The four ways it goes wrong
- Stale by default. The model's knowledge stops somewhere. Regulation does not. Transition periods end, amendments land, guidance gets replaced. The answer will describe the world as it was, with no marker that anything moved.
- Averaged across jurisdictions. Ask about a rule that exists in slightly different forms in several markets and you will get a blend that is true nowhere. The blend reads beautifully.
- Exceptions get flattened. Summaries remove exceptions because that is what summaries do. In regulatory work the exception is frequently the entire answer.
- Nothing to check. Even when the answer is correct, a professional cannot act on it, because they have to be able to show where it came from. "The model said so" is not a line you can put in a submission.
Why retrieval alone does not fix it
The obvious fix is to give the model the documents: retrieve the relevant passages and let it answer from those. That is a real improvement and it is where most teams stop. But I have seen it fail in quieter ways.
If the corpus contains the old and new version of a guidance document, retrieval will happily hand the model both, and the model will pick whichever reads more naturally. If a passage is chunked in the middle of a scope statement, the exclusion at the end of the paragraph never makes it into the prompt. If nobody tracks which document is authoritative for which market, the system is confidently wrong in a way that is now harder to spot, because it comes with a citation.
The source has to be in the loop, but it has to be in the loop properly: versioned, dated, scoped to a jurisdiction, and visible to the person reading the answer.
What working systems look like
The setups I have seen hold up in front of actual specialists share a few habits, and none of them are exotic.
- The document is the unit, not the chunk. Every answer points back to a specific document, version and section, and the reader can open it in one click.
- Dates are first-class. Publication date, entry into force, end of transition. The system knows them and shows them, so a stale answer looks stale.
- Jurisdiction is a filter, not a hope. You ask about one market and you get sources from that market. Blending is off unless you turn it on.
- Uncertainty stays visible. If the sources disagree or the scope is unclear, the answer says so instead of picking a side and sounding sure.
- A specialist reviews the workflow, not just the output. The people who do this job for a living are the only ones who know which edge cases matter. They belong in the design, not in the acceptance test.
The role AI should actually play
Once the source is properly in the loop, the model stops being the authority and becomes what it is genuinely good at: leverage. It finds the relevant passages faster than a human can scroll. It compares two versions and lists what changed. It drafts the first paragraph of the assessment so the specialist can spend their time on the judgment instead of the typing. It remembers which documents you have already reviewed and which ones are new since last month.
That is a large amount of value, and none of it requires the model to know the regulation by heart. It requires the model to read carefully, cite honestly and admit when the text does not say. Build for that, keep the documents where the reader can see them, and the fluent, confident, wrong answer stops being a risk you have to explain to a regulator.