AI and Information Governance: How Organizations Can Avoid Privacy Risks

Artificial intelligence is already a ubiquitous part of our lives. Whether you realize it or not, you are already interacting with AI. It may help you pick a restaurant on Yelp or guide you to a show to watch on Netflix. Similarly, AI is transforming how organizations collect and manage data. The converging worlds of AI and information governance pose significant privacy risks that legal teams should evaluate carefully.

Advantages of AI in Mitigating Privacy Risks

With the increasing volume of sensitive data, ensuring data security and privacy has become a top priority for organizations. Leading cybersecurity firms have already incorporated AI-driven tools to detect and respond to potential threats in real time, minimizing the risk of data breaches, a major contributor to the leakage of private data.

These systems use behavioral analysis and anomaly detection to identify suspicious activities, providing a proactive approach to cybersecurity. Additionally, AI can help organizations comply with data protection regulations by automating the identification, deletion, and management of sensitive information. These capabilities are also incorporated into Data Loss Prevention (DLP) tools to reduce the risks of private information leaving the organization.

Privacy Risks Associated with the Use of AI

When integrating AI into an organization’s infrastructure and operations, several key privacy considerations must be addressed to ensure responsible and ethical use:

1. Data Collection, Consent, and Leakage

AI systems often require vast amounts of data for training and operation. It’s crucial to obtain informed consent from individuals whose data is being collected and used. Organizations must be transparent about the types of data collected and how it will be used, especially if it may be used by an AI tool for training purposes.

Detailed information on how data is used can be exceedingly difficult to determine if that data goes into an AI tool, especially one that is publicly available. One study released in February found that publicly facing tools like ChatGPT, Microsoft’s Copilot, and Google’s Gemini are both widely used and are a major source for leaking sensitive data. They found and documented numerous examples of sensitive and regulated data being regularly leaked.

In another example with a team I worked with, we used actual medical records that we had anonymized to perform a query. Specifically, we asked for diagnoses for patients that displayed similar medical conditions as those in the records we submitted. Not only did we get a series of answers with specific diagnoses, in some cases we actually got patient names, facility/location name, and doctor/med tech names for several records for patients that were not part of our data set. The Protected Health Information (PHI) for those patients was leaked by the AI tool we were testing, having been entered into the system by someone else.

2. Data Security

Protecting sensitive information from unauthorized access is paramount. AI systems should incorporate robust security measures, such as encryption and access controls, to safeguard data against breaches and leaks.

Even if your internal security environment is secure, it is vital to understand the security risks to privacy that third parties can present. Your payroll vendor, for example, likely has substantial amounts of sensitive information and other PII on your employees. It is important to understand how they use AI and to limit what they can do with your data. Similarly, ensure that you have a good understanding of how any official AI vendors your organization uses or vendors that incorporate AI into their offerings protect the any data they have access to.

3. Regulatory Compliance

Adhering to privacy regulations, such as the European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), is essential. Organizations must ensure that their AI systems comply with relevant laws and standards to protect individual privacy rights.

GDPR, CCPA, and other privacy regulations also have specific requirements for how data is managed that have implications for the proper use of AI. For example, GDPR has specific requirements around transparency of processing and use of an individual’s data. This means that you must be able to articulate the purpose of data collection, how the data will be used, and what the impact may be. This means that you need to understand how any AI tool will use any sensitive data prior to submitting that data.

These regulations also often require that you get explicit consent before processing a person’s data, with that consent being informed, specific and revocable. Revocation of consent means that individuals must have the right to access their personal data and to request its deletion. It may be extraordinarily difficult to determine how third-party AI systems use data and even more difficult to get that specific data deleted.

AI is likely to be as transformative as the internet itself has been for how we live our lives and conduct business. But just like the internet, AI and information governance practices pose risks and unintended consequences. It is vital to understand those risks, especially the privacy risks, to mitigate the risk to your organization.