AI Governance Framework: How to Build One From the EU AI Act

What Is an AI Governance Framework?

An AI governance framework is an organization's operating structure for AI: who owns each system, which systems exist, how risky each one is, which controls apply to it, and what evidence shows the controls worked. It sits between the external texts that bind or guide you and the internal policies your teams follow.

Several kinds of text are called frameworks, and they differ in force. The OECD says of its own instruments that "Recommendations are adopted by Council and are not legally binding", in the notes to its Recommendation of the Council on Artificial Intelligence, while the EU AI Act imposes legal duties on providers and deployers.

The framework is the internal layer that maps these texts to owners and evidence. The techniques used to check that an AI system meets those rules, from audits to performance testing, are the subject of the AI assurance guide.

Which AI Governance Frameworks Apply to Your Organization?

The AI governance frameworks below were read on official pages on October 20, 2026. Which ones bind you depends on where you sell, whether you build or deploy AI, and what your customers write into contracts.

• EU AI Act dates - the Commission's AI Act policy page says the Act entered into force on August 1, 2024 and "became applicable on 2 August 2026, with some exceptions": prohibitions 1 to 8 and AI literacy duties have applied since February 2, 2025, general-purpose AI rules since August 2, 2025, a ninth prohibition added by the AI Omnibus applies from December 2026, and the high-risk rules come later (next item).
• Digital Omnibus on AI - the European Parliament's Legislative Train records Regulation (EU) 2026/1744, signed on July 8, 2026, which moves the high-risk duties of stand-alone systems to December 2, 2027 and those of systems embedded in regulated products to August 2, 2028. Guidance dated before July 2026 may cite the earlier dates.
• NIST status - the NIST AI RMF page calls the framework "intended for voluntary use", says version 1.0 is under revision through the White House AI Action Plan, and lists a concept note for a critical infrastructure profile, released on April 7, 2026. The guide to the AI risk management framework covers its four functions, TEVV and the Generative AI Profile.
• ISO certification - ISO lists ISO/IEC 42001 as edition 1, published in 2023-12, and ISO/IEC 42006:2025 "sets out the additional requirements for bodies that audit and certify artificial intelligence management systems (AIMS) according to ISO/IEC 42001", which makes 42001 the one text in the table that an organization's management system can be certified against.
• Council of Europe ratifications - Article 30 of the Framework Convention text brings it into force on the first day of the month after a three-month period that starts once five signatories, including at least three Council of Europe member states, have consented to be bound. The Council of Europe's own chart of signatures could not be opened on October 20, 2026 (it answered HTTP 403), so the status comes from the Dutch government's treaty database, read that day, which lists 21 signatories and one ratification, by the European Union on May 15, 2026, and no entry-into-force date.
• Singapore frameworks - the PDPC calls the Model AI Governance Framework, second edition a "living and voluntary Model Framework", and IMDA's Model AI Governance Framework for Agentic AI, version 1.5, is "targeted at organisations looking to deploy agentic AI, whether by developing AI agents in-house or using third-party agentic solutions."

The table leaves out sector rules such as banking model risk guidance and privacy law, which still apply to AI systems in their scope. Name them in your framework next to the AI-specific texts, with the same owner and evidence columns.

How Do the Major AI Governance Frameworks Compare on What They Require?

Reading the texts side by side shows where each puts its weight. A script run on October 20, 2026 counted terms for seven obligation types per 10,000 words in six official texts: the EU AI Act as published in the Official Journal, NIST AI 100-1, and the OECD, Council of Europe and Singapore texts linked earlier in this guide. The figures in this section are the script's output.

For the EU AI Act, the script counted the articles and annexes of the Official Journal text only, because recitals do not bind, and that copy predates the Digital Omnibus amendments. PDFs were converted to plain text with pdftotext before counting.

ISO/IEC 42001 is not in the matrix: ISO sells the full text, and its public abstract is too short to count. The fixed term list, matched without regard to case:

The script's output, without its raw-count table:

Obligation terms per 10,000 words
Run 2026-10-20, official texts, fixed term list

              EUAIA    RMF   OECD    CoE   MGF2   AGNT
Words         52672  15120   4532   4724  15925  17585
Risk mgmt         5     54      7      4      6      9
Data              5     10      4      0     37      7
Logs              5      1      4      0     13     13
Transparency      5     34     18      4     30      9
Oversight         2      5      4      0     10     28
Testing          30     27     13      6     28     54
Monitoring       18     19      9      2      6     28

EUAIA = EU AI Act
RMF   = NIST AI RMF 1.0
OECD  = OECD AI Recommendation
CoE   = CoE Convention CETS 225
MGF2  = Singapore MGF, 2nd ed.
AGNT  = IMDA MGF Agentic v1.5

EUAIA Testing hits in the phrase
'testing in real world conditions': 54

• IMDA's agentic framework - the highest rates for testing (54) and human oversight (28) in the run, which fits a text about AI that takes actions.
• NIST AI RMF - the highest risk management rate (54), more than ten times the EU AI Act's 5, because the whole framework is organized around managing AI risk.
• Singapore's 2020 model framework - the highest data rate (37), well above the EU AI Act's 5.
• EU AI Act - rates of 5 or fewer for risk management, data, logs, transparency and oversight, because much of its 52,672 words of articles and annexes goes to other subjects, such as its chapters on governance, market surveillance and penalties. Its testing rate (30) is the second highest in the run, and 54 of its 156 testing hits sit in the phrase "testing in real world conditions", which the Act uses for its rules on real-world testing, among them Article 60 on testing high-risk systems outside regulatory sandboxes.
• Council of Europe convention - zero hits for the listed data, log and human oversight terms, because it addresses states: its Article 16 has each Party "adopt or maintain measures for the identification, assessment, prevention and mitigation of risks" and leaves the detail to national law. The convention is not silent on data, though: its Article 11 covers privacy and personal data protection.

Term density shows what a text talks about, not how hard it binds: a single paragraph of the EU AI Act's record-keeping article creates a legal logging duty for high-risk systems, while a voluntary text can mention logs often and bind no one. Use the matrix to pick which text to read first for each obligation, then write the control from the text that binds you.

Which Layers Does an AI Governance Framework Need, and in What Order?

Build it layer by layer, in order, because each layer depends on the one before it: you cannot tier systems you have not listed, or test controls nobody owns. Each layer below names the artifact an auditor or customer will ask for and the framework text behind it.

How Do You Write a Governance Policy Line as a Test Criterion?

Rewrite each policy line as a criterion a test can decide, with the evidence it needs named beside it, run the system against inputs built to test the rule, and record one of three results: Pass, Fail or Unable to Verify. The third result exists because missing evidence is not compliance, an idea close to NIST's Measure function, which asks that risks that "will not" or "cannot" be measured be "properly documented" rather than assumed away.

The first policy in the table below comes from Article 50 of the EU AI Act, under which providers ensure that people are "informed that they are interacting with an AI system"; the others are typical internal rules.

Written out for a test tool, the logging policy splits into two criteria, each with its own evidence and its own Unable to Verify condition. The format below is tool-neutral, and it uses 184 days, the longest span six calendar months can cover (July through December):

policy: LOG-04
text: >-
  Each AI decision is logged with its inputs and
  model version, and kept at least six months.
criteria:
  - id: LOG-04-a
    statement: >-
      A log record for the test decision holds the
      input, output, model_version and timestamp.
    evidence: read the log store by decision_id
    pass: all four fields present and non-empty
    fail: no record, or a field is empty
    unable_to_verify: the test cannot read the store
  - id: LOG-04-b
    statement: Log retention is 184 days or more.
    evidence: retention setting of the log store
    pass: setting is 184 days or more
    fail: setting is below 184 days
    unable_to_verify: setting not readable

• Aim inputs at the rule - for the fee policy, ask about a fee that changed last quarter, since a stale answer only shows up there.
• Keep Unable to Verify out of the pass rate - report it as its own count with the missing evidence named, so a dashboard cannot turn unread logs into a green result.
• Close gaps in the system - many Unable to Verify results close once the system logs more, such as the model ID or the schedule version with each reply.

What Changes in AI Governance When an AI System Can Act?

When an AI system can call tools, change records or send messages, its risk depends on what it can do as well as what it says. IMDA's agentic framework ties that risk to "the scope of actions the agent can take, the reversibility of those actions, and the agent's level of autonomy", and adds that "changes to one agent can have cascading effects across interconnected workflows."

• Inventory - record each agent's tools, credentials and write access, since those set what it can change.
• Risk tiers - weigh how reversible the agent's actions are; sending a payment and drafting an email belong in different tiers even inside one use case.
• Controls - check the changed record and the tool calls with their arguments, not the agent's own report of what it did.
• Rollout - IMDA notes that agents' autonomy and changing environments make it "challenging to account for and test all possible outcomes before deployment", and recommends gradual rollout with monitoring.

Oversight duties also apply to the actions. For high-risk systems, Article 14 of the EU AI Act requires that the people overseeing a system can "disregard, override or reverse the output"; for an agent, plan a way to block or undo an action as well as to ignore a reply. Agent-specific controls and audit trails are covered in the guide to agentic AI governance.

How Do You Measure Whether AI Governance Is Working?

Measure governance by the evidence it produces. For high-risk systems, Article 72 of the EU AI Act asks providers for a post-market monitoring system whose data lets them "evaluate the continuous compliance of AI systems" with the high-risk requirements, and the same habit tells you whether your framework holds.

Report the unverifiable share next to the pass rate every time. A system that passes most of what could be checked, while most of its criteria could not be checked, has weak evidence, and the fix often starts with what the system logs.

Running Governance Policy Tests on AI Agents in Agent Assurance

TestMu AI's Agent Assurance, built for AI agents that act, turns your written policies into acceptance criteria like the ones above and grades them against what the real agent did, before it ships. The product is at pre-alpha, publicly installable, and its terminal client is Rook CLI:

npm install -g @testmuai/rook

For the npm route, have Node.js 20 or newer in place. Rook CLI runs on macOS and Linux, and 64-bit Windows through npm or WSL; Homebrew and a checksum-verifying shell installer cover macOS and Linux and need no Node.

• Policies as input - it reads the agent's codebase, or a folder holding a PRD and a policies file when you cannot share code; the guide to connecting and exploring agents notes that a documentation-only exploration has less evidence about tool behavior and side effects.
• Scenarios with criteria - generated scenarios carry their own acceptance criteria, and the default classes are functional and adversarial.
• Evidence from the run - a criterion can be decided by the tool calls set against the agent's declared tools, by files the run changed in the paths its profile declares, or by a record looked up with a read-only MCP tool, which you provide and sign off. Model judges score some criteria, but an agent's claim that it acted is never taken as proof.
• Three verdicts - the result for a criterion is Pass, Fail or Unable to Verify, and Unable to Verify results stay out of the pass rate's denominator, counted neither for nor against the agent.
• Records kept per run - each run saves plain files such as run.yaml, report.yaml and a verdict.yaml per scenario; the Agent Assurance results and evidence docs list the full set.

Most eval and observability tools grade an agent on its replies and its recorded traces, while Agent Assurance judges an agent by the changes a run leaves behind and names what it could not verify.

Give it a staging copy of your systems and synthetic records to work on: what the agent writes during a test stays written. It does not watch production or block anything at runtime, and its target is the agent you built, not the model it calls.

Agent Assurance does not map to, comply with or certify against the EU AI Act, ISO/IEC 42001, the NIST AI RMF or any other framework in this guide. It tests and reports; whether your agent meets the EU AI Act, ISO/IEC 42001 or an internal policy is for your compliance team and auditors to decide.

Start Your AI Governance Framework From the Inventory and the Highest-Tier System

Start with the inventory: list every AI system in production with its owner and model version, then tier each one. Take the highest-tier system, rewrite three of its policy lines as acceptance criteria in the format above, and run them.

The criteria that come back Unable to Verify are the first gaps your AI governance framework has to close. For an agent that changes records or calls tools, the same criteria can run against staging through Agent Assurance once Rook CLI is installed by following the Rook CLI installation guide.

Note: AI assistance was used in researching and drafting this article. Sirajuddin Khan (Vice President of Product Management at TestMu AI, expertise in agentic AI and GenAI product strategy) wrote it as author of record, with Vipul Verma, Group Senior Vice President of Engineering at TestMu AI, as technical reviewer. Before publication, each statistic, link and product claim was checked against primary sources: the official texts and pages of the European Union, NIST, ISO, the OECD, the Council of Europe, Singapore's PDPC and IMDA, and the Dutch government's treaty database, plus TestMu AI's product documentation, all used following our editorial process and AI use policy.