- Start with an inventory. You cannot govern AI you don't know about — including AI built into the SaaS tools you already use.
- Tier every use case by risk and scale the controls to the tier. Most internal use cases need light controls.
- NIST AI RMF tells you what to think about, ISO/IEC 42001 tells you how to run it as a management system, and the EU AI Act tells you what is legally required.
- Evidence matters as much as policy: keep evaluation results, decision logs, and approvals where an auditor can find them.
Two years ago, AI governance was a topic for large banks and big tech. Today it shows up in ordinary security questionnaires, enterprise contracts, cyber-insurance renewals, and board meetings. Buyers want to know which AI systems touch their data, how outputs are checked, and who is accountable when something goes wrong.
The good news is that governing AI well does not require a new department. It requires a clear inventory, sensible risk tiers, controls that match those tiers, and evidence that the controls work.
Why AI governance can’t wait
- Regulation is arriving in stages. The EU AI Act entered into force in August 2024, with bans on prohibited practices from February 2025, obligations for general-purpose AI models from August 2025, and most high-risk requirements phasing in from 2026 onward. In the US, state laws such as Colorado’s AI Act and New York City’s rules on automated hiring tools add their own requirements.
- Customers are asking. Enterprise buyers now include AI-specific questions in vendor reviews. Having clear answers shortens sales cycles.
- Shadow AI is already here. Employees use public AI tools whether or not there is a policy. Governance is how you channel that into approved, safe tools.
The three frameworks, compared
| Framework | What it is | Best used for |
|---|---|---|
| NIST AI RMF | Voluntary US framework built around four functions — Govern, Map, Measure, Manage — with a companion profile for generative AI (NIST AI 600-1) | Structuring how you identify and reduce AI risks |
| ISO/IEC 42001 | International, certifiable standard for an AI management system, published in 2023 | Running governance as a repeatable, auditable process — and proving it to customers |
| EU AI Act | Law that sorts AI systems into prohibited, high-risk, limited-risk (transparency), and minimal-risk categories | Knowing what is legally required for systems used in the EU |
These fit together rather than compete. A practical programme uses NIST AI RMF to think through risks, runs the process the way ISO/IEC 42001 describes, and checks each system against the EU AI Act and any US rules that apply. As an ISO 27001-certified company, we have found that organisations with an existing information security management system can reuse much of that structure.
The six-step playbook
- Inventory every AI use. Custom models, LLM integrations, vendor tools, and AI features inside existing SaaS products. Record the owner, purpose, data used, and who is affected.
- Tier each use case by risk. Consider who is affected, what decisions the system influences, what data it touches, and how easily a mistake can be caught and reversed.
- Publish an acceptable-use policy. Which tools are approved, what data may never be entered, and how to request a new use case. One or two pages is enough.
- Apply controls by tier. Data protection, human oversight, testing, logging, and vendor review — scaled to the risk, as shown below.
- Keep the evidence. Evaluation results, model and data documentation, approvals, incidents, and decision logs, stored where auditors can find them.
- Monitor and review. Re-test after model or prompt changes, review the inventory quarterly, and track incidents to closure.
Controls by risk tier
| Tier | Examples | Minimum controls |
|---|---|---|
| Low | Drafting internal emails, summarising public documents, code suggestions | Approved tools only, no confidential data in public tools, user training |
| Medium | Internal knowledge assistants, document extraction with human review, support drafting | Private deployment, access controls that mirror source permissions, logging, evaluation before release |
| High | Decisions about people — claims, credit, hiring, care — or actions taken without review | Documented risk assessment, bias and accuracy testing, human approval for adverse outcomes, explainability, full audit trail, regular independent review |
Keeping governance lightweight
The governance programmes that work are the ones teams actually use. A short intake form for new AI use cases, a fast approval path for low-risk work, and deeper review only where the risk justifies it will do more than a long policy nobody reads.
Governance should also make building easier, not harder. When the approved patterns — a private LLM endpoint, a standard logging setup, a reusable evaluation harness — already exist, teams pick the safe path because it is the fastest one. Our guide to private AI and RAG describes one such pattern.
Frequently asked questions
Do we need ISO/IEC 42001 certification?
Not necessarily. Certification makes sense when customers or regulators ask for independent proof, or when AI is central to your product. Many companies start by aligning their practices with ISO/IEC 42001 and NIST AI RMF, then certify later. If you already hold ISO 27001, much of the management-system structure carries over.
Does the EU AI Act apply to US companies?
It can. The Act applies to companies that place AI systems on the EU market or whose AI outputs are used in the EU, regardless of where the company is based. Obligations depend on the risk category of each system, and several deadlines have been phased in or proposed for adjustment, so confirm the current timeline with legal counsel.
What is shadow AI and why does it matter?
Shadow AI is AI use the organisation does not know about — employees pasting data into public chatbots, or AI features switched on inside existing SaaS tools. It matters because it is where confidential and personal data most often leaks. An AI inventory and a clear acceptable-use policy are the first defence.
Who should own AI governance in a mid-size company?
A small cross-functional group works best: a senior business sponsor, security or compliance, legal or privacy, and a technical lead who understands how the AI systems are built. It should meet regularly and own the AI inventory, the risk tiers, and approvals for new use cases.