Doing AI Governance | Edition #58: Fixing broken AI governance AND Triaging AI governance problems


This newsletter provides practical guidance, tools and resources for the real work of governing safe, secure and lawful AI.

Newsletter #58 - June 2026

Feature Article #1

How I triage AI governance problems

By James Kavanagh


You get hired to review one AI system. Within hours you find a dozen, and the one everyone's worried about turns out to be the safest of the lot. The systems that should worry you are usually the ones nobody can see. This is the method I use to work out where to start.

Most people imagine the hard part of AI governance is the technical work. Classifying a system under the EU AI Act. Auditing against ISO 42001. Building the controls. The harder problem usually shows up before any of that, on your first day. You walk in to review one system and discover there are twelve. Some were bought, some built in-house. Others drifted from a research prototype into operational decision-making one reasonable step at a time, and nobody noticed. Most of them have no governance at all.

So where do you start? You have limited time, no structure, and a stack of systems that all feel urgent. That's a triage problem.

In this article I walk through the approach I take. It leans on the work of Jens Rasmussen, the safety scienitist who showed that organisations rarely fail because someone does something obviously dangerous. They fail because a lot of small, reasonable decisions move the system toward a line nobody can see. Governance attention flows to the systems people can see, which means it often ends up inversely proportional to the real risk.

The method has four steps. Work out who is actually affected. Surface the harms each group could experience. Characterise each harm across three dimensions - impact, severity and detectability. Severity times likelihood is the risk matrix everyone knows, and it misses a simple question. Would you even know this was happening? A severe harm that's obvious triggers a response. A severe harm that's invisible just accumulates.

If you've ever stared at a list of AI systems and not known where to begin, start here.

This kind of risk prioritisation is the work we cover in the AI Governance Foundation Track and AI Compliance Speciality course. Join the waitlist.

The AI Governance Practitioner Program

Foundation Track: Practitioner Cohort #2

US$899. Spots filling fast.

Cohort #1 is fully subscribed and kicks off this month. Cohort #2 starts 25 August and is now open.

Eight weeks. A group of no more than 15 practitioners. You work through all four Foundation Track courses online in fortnightly live sessions with me (James Kavanagh), with a dedicated peer community, and direct email access throughout. We go deep on the real-world practices and case studies, and you'll learn to build governance within the VerifyWise GRC platform.

Complete a graded assignment and interview at the end and earn your Practitioner Award, AI Governance Foundations, included in the cost.

Cohort #2 starts 25 August 2026

Feature Article #2

How I fix broken AI governance

By James Kavanagh


Once you know which system to work on, what do you actually do? Most governance assessment stops at “there's a gap,” which isn't a diagnosis. This is the method I use to find what's really broken, and to fix it without launching a transformation program nobody needs.

There's a story about Taiichi Ohno, the engineer behind the Toyota Production System. He used to draw a chalk circle on the factory floor, make a manager stand in it, and walk away. The manager's only job was to watch. To see how work actually flows, where it gets stuck, and where the procedure posted on the wall bears no resemblance to what people are really doing.

You can't improve a system you haven't observed, and what you observe won't match what you're told.

That's where the real work of AI governance starts. It doesn't start with mapping the EU AI Act or writing a policy. It starts with standing in the circle and watching. Once the triage method tells you which system to work on, the next question is what's actually broken, and what kind of fix it needs.

In this article I walk through my diagnostic method. I'm hard on the usual approaches, because most of them stop short of anything useful. Maturity models hand you a score without a diagnosis. Gap analysis tells you a control is missing but not why. GRC dashboards track whether someone uploaded a document, not whether the mechanism behind it works. Questionnaires give you what people believe happens, not what they'd see if they stood in the circle and watched.

What's missing is a way to get underneath the control. I do that by asking whether a mechanism even exists, whether it's working, what specifically is wrong with it, and what kind of intervention each problem needs. That last question is where I see most people go wrong. They throw a tool at a problem that's really about how people think, or run training to fix something that's really a tooling gap. Matching the right fix to the right problem is the real skill.

If you want governance that keeps working as the technology and the rules keep shifting, this is how I'd begin.

This diagnostic work sits at the center of our AI Compliance Specialty Course, launching in July, with AI Risk close behind. Join the waitlist.


FREE AI Governance Assessment

Practitioner Capability · Organizational Capacity

Hundreds of practitioners have now taken the assessment. It's free, takes about 10 minutes, and your results are emailed immediately.

Take the FREE Assessment

We're running two live webinars in June to walk through what your results mean and how to prioritise what's next for you. We cover the same content in both sessions, just choose the session that is best for your time zone.

Webinar #1: Wed 24 June, 10:00 AM AEST (Tue 23 June, 5:00 PM Pacific · 8:00 PM Eastern)

Webinar #2: Thu 25 June, 11:00 AM UK · 12:00 PM Central Europe · 8:00 PM AEST

Register for a June webinar


What we're working on

Our AI Compliance Specialty Course is coming along nicely and is on schedule for a July release date. Right now, we're focusing on the Balcony tool you'll use throughout the course. We will go out to our waitlist with a special offer prior to the course being made available to the general public, so be sure to join now. Join our specialty course waitlists here.

And for those of you who have completed, or are working through, the Foundation Track, we'll be in touch over the coming weeks with more information on our new credential - the Practitioner Award, AI Governance Foundations. Learn more about AI Career Pro Practitioner Awards and other credentials here.

As always, reach out if you have questions or need further clarity on any of our work, or if you need help navigating your professional path. We're learning with you and really appreciate your enquiries and feedback.

PS. You're receiving this as a subscriber to communications from AI Career Pro. We respect your privacy, so please unsubscribe through the link below if you do not wish to receive these communications in the future.

PO BOX 7087, Redhead, NSW 2290
Unsubscribe · Preferences

Doing AI Governance

Join over 4,500 subscribers and learn about the real work of AI governance. Moving beyond theory, we focus on the practical application of AI governance in real-world organisations with case studies, tools, templates and guidance. Led by James Kavanagh - the AI governance practitioner who led governance at both AWS and Microsoft.

Read more from Doing AI Governance

This newsletter provides practical guidance, tools and resources for the real work of governing safe, secure and lawful AI. Newsletter #60 - July 2026 Feature Article #1 Building an AI Inventory is your First Governance Intervention By James Kavanagh Your first meaningful governance intervention is most likely building an AI inventory. It's the first time anyone in the organisation has to sit down and agree, out loud, on what AI is really running and what it's for. That's not an admin task...

This newsletter provides practical guidance, tools and resources for the real work of governing safe, secure and lawful AI. Newsletter #59 - June 2026 Feature Article #1 How I scope compliance obligations for AI governance By James Kavanagh The biggest compliance obligations on your AI systems rarely come from AI regulation. Here's how I work out what actually applies, starting from what the business does, not the law. When practitioners new to AI Governance start scoping their compliance...

This newsletter provides practical guidance, tools and resources for the real work of governing safe, secure and lawful AI. Newsletter #57 - May 2026 Feature Article A new path to demonstrated practitioner capability in AI governance. By James Kavanagh Law has supervised practice before admission to the bar. Medicine has residency. Engineering has years of oversight by senior engineers before obtaining a charter. Each serious profession recognizes that capability builds in stages, and that...