AI Governance for Mid-Size IT: The Lightweight Operating Model

Search for an AI governance framework and you’ll find material written for large enterprises: dedicated AI ethics offices, risk committees with dozens of members, model validation teams, documentation requirements that would take a mid-size IT team a year to satisfy. It’s not wrong. It’s just built for organizations with far more people than you have.

Mid-size organizations still need governance, and arguably need it more, because they have fewer specialists to catch problems. What they need is a lightweight operating model: a small number of components, clear owners, a steady cadence, and a few hours of effort a month once it’s running. This post lays one out on a page and maps it to the NIST AI Risk Management Framework, so you can show auditors and customers it rests on a recognized standard.

The four governance questions in the assessment

The governance dimension of the AI Readiness assessment asks four questions. Each has its own guide in this series.

1. Policy: do you have an AI acceptable use policy? Published and trained beats perfect and drafted. See publish your AI acceptable use policy this month.
2. Decision rights: who decides which AI tools and projects get approved? A small cross-functional council with an intake process. See the 30-minute AI council.
3. Regulation: have you mapped the rules that apply to you? Before the pilot, not after. See SOX, state privacy laws, and AI.
4. Inventory: do you know what AI is in use, including inside SaaS? It’s the foundation everything else relies on. See build your AI inventory in two weeks.

The pattern I see most often

Governance is usually the lowest-scoring dimension in the mid-size assessments I run, and the profile is remarkably consistent. The acceptable use policy is “being drafted.” IT approves AI tools by default, because nobody else has been asked to. Legal gets consulted when someone happens to think of it. And the AI inventory is a list someone could produce from memory if asked. None of that reflects a lack of care. It reflects the fact that nobody has been given the job. The operating model below is mostly about giving the job to someone and making it small enough to actually do.

The five components of a lightweight operating model

1. A policy that tells people how to use AI, in two pages or less, with named approved tools and three data tiers.
2. A decision forum: a council of five or six people meeting thirty minutes a month, with an intake form and a fast track for low-risk requests.
3. An inventory of every AI tool and feature in use, with owners, data types, and risk ratings.
4. An obligations map that shows which laws, contracts, and industry rules apply to each high-risk use, and which control meets each one.
5. A reporting rhythm: a quarterly one-page report to leadership on value, risk, and decisions.

That’s it. Each component produces one artifact, and each artifact has one owner. Anything more elaborate should wait until you have a specific reason for it.

Who runs it

• Executive sponsor: chairs or delegates the council, owns the quarterly report, and resolves cross-department disputes.
• AI program owner: usually the IT leader or someone they designate. Runs the intake, maintains the inventory, prepares the council agenda, and makes sure the cadence happens. This is the role that makes or breaks the model, and it needs a few hours of protected time a month.
• Council members: IT, security, legal or compliance, HR, and business representatives.
• Business owners: own individual use cases and their results.
• Champions: one per department, feeding ideas and questions in.

The cadence

• Weekly: fast-track decisions on low-risk requests by the program owner.
• Monthly: the thirty-minute council meeting.
• Quarterly: the leadership report, a re-scan of systems for new AI tools, and a review of high-risk inventory items.
• Twice a year: policy review.
• Yearly: a review of the operating model itself: what’s working, what’s become ceremony, and what to add.

Mapping it to the NIST AI RMF

The NIST AI Risk Management Framework is voluntary, free, and organizes AI risk work into four functions. The lightweight model covers each one:

• Govern: the policy, the roles, and the council. This is the function that establishes accountability.
• Map: the inventory and the obligations map, which establish what AI is in use, in what context, and what rules apply.
• Measure: risk ratings, the metrics you track for each use case, and the monitoring described in observability for AI.
• Manage: council decisions, incident runbooks, and retiring tools that no longer justify their risk.

Stating this mapping in your policy or council charter is a small step with a real payoff: when a customer questionnaire or an auditor asks which framework you follow, you have a credible answer and the artifacts to back it up. If customers later require certification, ISO/IEC 42001 is the certifiable standard, and a working NIST-aligned model is a good foundation for it.

The asset: governance on a page

Put this table on one page and share it with leadership. One row per component:

• Component: policy, decision forum, inventory, obligations map, reporting.
• Artifact: the document or list it produces.
• Owner: a named person.
• Cadence: how often it’s reviewed or produced.
• NIST function: Govern, Map, Measure, or Manage.
• Status: not started, in progress, or running.

When every row says “running,” you’re at or near level 4 on all four governance questions.

A 60-day setup plan

Days 1 to 15: name the executive sponsor and the AI program owner. Adapt and approve version 1 of the acceptable use policy in a single review meeting. Stand up the intake form.

Days 16 to 30: hold the first council meeting to approve the charter and triage rule. Publish the policy with training. Start the two-week inventory build.

Days 31 to 45: finish the inventory and risk-rate it. Run the pre-pilot regulatory screen on every high-risk item and start the obligations map.

Days 46 to 60: hold the second council meeting to review the inventory and pick the next pilots from the use-case register. Draft the first quarterly report.

Mistakes I see at this stage

Copying an enterprise framework. Heavy governance at mid-size scale stops within a quarter. Start light and add only what a real need justifies.

No program owner. Governance with no one responsible for running the cadence quietly stops.

Governance as a brake. If the model mostly says no, people route around it. A fast track for low-risk requests is essential.

Documents without decisions. A policy and a charter mean little if the council never approves or rejects anything. The decision log is the proof governance is working.

How governance fits with the other five dimensions

Governance is one of six dimensions in the AI Readiness framework, and it’s the one that ties the others together: it decides which tools are approved, which use cases proceed, and how risk is tracked. For the whole framework, see the 6-dimension AI readiness framework, explained.

Where does your team actually stand?

The free AI Readiness Score includes governance questions alongside the other five dimensions. It’s 10 questions and gives you a score in a few minutes.

Get your free AI Readiness Score →

Want to see what the full assessment covers first? Flip through a complete 38-page sample report.

Related guides

• Publish Your AI Acceptable-Use Policy This Month (3-Tier Template)
• The 30-Minute AI Council: Lightweight Governance That Sticks
• SOX, State Privacy Laws, and AI: Map Obligations Before the Pilot
• Build Your AI Inventory in Two Weeks (Auditors Will Ask)
• Observability for AI: Logging What Your Assistants Do