Don’t Overcomplicate Your AI Governance, Here’s a Better Approach

Your CEO has declared that the company will use AI across the organization, and you experience a flash of panic. How do we know our use of AI will be compliant? Have we identified the risks? Will we inadvertently leak sensitive information? Will using AI create new risks? Your next move may well determine the success of your AI governance, and possibly your company’s overall use of generative AI. Consider it carefully.

Don’t start by creating an overarching program

Your immediate instinct may be to research and adopt global AI governance frameworks, standards, and compliance rules for an overarching, enterprise-wide AI governance program. Doing so may be a mistake, as you will likely be setting your organization up for months of paralysis while the business moves forward without you.

We see the same pattern play out repeatedly as companies begin by trying to build a top-down governance program around a framework, surveying the European Union’s AI Act, the emerging patchwork of US state laws, and competing global frameworks from the National Institute of Standards and Technology (NIST), Massachusetts Institute of Technology (MIT), and International Organization for Standardization (ISO) to create a comprehensive AI policy. Quickly, a question arises: should the AI governance policy handle personal information management, or should the current privacy policy address it? The same question arises around the data security classification policy, records retention policy, ethics policy, etc. The governance effort quickly bogs down, while the pressure to use AI increases.

A smarter path begins with a simple question: what are we actually doing with AI?

Don’t try to begin by building governance around what AI might look like in two years. Instead, start with the specific use cases your organization has today and establish a strong foundation for future growth.

Govern by use case

Most organizations’ AI use falls within a spectrum from basic to sophisticated. Understanding where your organization sits, and governing accordingly, is more effective than constructing an enterprise-wide program in the abstract.

At the entry level, employees are already using AI chat tools such as Claude, ChatGPT, or Gemini, typically in an ad hoc and ungoverned way. The next level involves AI that accesses enterprise data, such as Microsoft Copilot or Google Workspace AI. Beyond that, many organizations are discovering that AI has been quietly embedded into third-party applications they already use, including contract management systems, ERP platforms, and HR tools. More sophisticated use cases include internally built AI applications, advanced systems that pull from proprietary data through a process called retrieval augmented generation (RAG), and finally, agentic AI that takes autonomous action on behalf of users.

Focus on readiness for each specific use case:

• Chat tools and ad hoc AI: For most organizations, AI chat tools are the true entry point into generative AI. Employees are using them whether governance is in place or not. The primary risk here is the leakage of sensitive information. When an employee pastes proprietary data into a public AI tool, that information can become part of the model’s training data. The governance requirements at this tier are relatively straightforward: an AI governance policy, a data security classification standard that helps employees understand what information can and cannot be shared, and basic AI training. The governance lift is light, yet this is often entirely missing.
• AI accessing enterprise data and third-party embedded AI: Tools like Copilot scan everything in your Microsoft 365 environment, including email, SharePoint, Teams, and OneDrive. Without the right sensitivity labels and access controls in place, an employee could inadvertently surface confidential salary information, pending transactions, or sensitive client data. A good records retention schedule, an updated data security classification standard, and properly implemented sensitivity labels are not just nice to have here; they are the foundation. For AI embedded in third-party applications, do not assume that because you have always used a vendor’s platform, their AI implementation is automatically governed appropriately. Ask how they are using AI, how they are accessing your data, and whether your data processing agreements include zero-data-retention terms where appropriate. Many legacy vendors are moving very quickly to embed AI capabilities, and the controls are not always applied by default.

• Internally built and agentic AI: When your organization builds its own AI applications, the transparency obligation rests with you as the developer. A Canadian court made this point clearly when an airline was held responsible for incorrect information provided by its own chatbot. At this tier, the governance checklist expands to include an AI development lifecycle process, human-in-the-loop controls, and rigorous testing and audit procedures. For agentic AI, which takes autonomous action such as booking travel, routing work, or updating systems, the stakes rise further still. If your organization is not yet using agentic AI, be aware of it, but focus your energy on the use cases you actually have today.

Policies, frameworks, and dashboard tools

What can get in the way? The following are examples of approaches that can overcomplicate AI governance efforts.

• A standalone AI policy. In most cases, the better approach is to strengthen existing policies around records management, data security, privacy, and ethics. Use an AI policy as a connector among them rather than a parallel document that risks conflict.
• A framework-first approach. Frameworks such as NIST are useful for identifying risk, but implementing a framework is not the same as governing AI use. First, start with what you are actually doing. Then pull in frameworks to help you ask the right questions.
• Compliance dashboard tools can help identify regulatory requirements by jurisdiction, but they typically do not address how specific use cases actually work, what data they touch, or whether your safeguards are operating effectively. Don’t believe vendors’ promises that their dashboard will be an “easy” button for governance.

Start somewhere specific

Organizations that are getting AI governance right are not necessarily the ones with the most elaborate programs. Instead, they are the ones that started with a specific use case, built repeatable processes around it, and expanded from there. Lessons learned from simpler use cases go a long way toward preparing you for more advanced ones.

If your CEO says the company will use AI for everything, your goal as legal counsel is to make sure the organization can say “yes” safely, compliantly, and with confidence. Starting with your actual use cases, and building governance from the bottom up, will help you get there quickly.