Generative AI for Document Review: Is It Accurate and Defensible?

Relativity’s Cristin Traylor reports that the viability of generative AI (GenAI) for document review is predicated on two major concerns: accuracy and defensibility. While GenAI is quickly emerging as a powerful tool for document review, legal teams face the same fundamental questions that accompanied the rise of technology-assisted review (TAR). 

Just as TAR became legally accepted through rigorous statistical validation, GenAI’s success will depend on applying those same metrics to ensure consistent and defensible results. Metrics such as recall, precision, and richness determine the percentage of documents in a population predicted to be relevant. The percentage of documents that are predicted not to be relevant is determined by elusion metrics. 

The advantage introduced by using GenAI for document review is timing: Validation can occur before a full review begins. By running smaller, statistically significant tests, legal teams can refine prompts with input from subject-matter experts, identify weaknesses, and strengthen outputs without incurring the cost of a full-scale rerun.

This proactive validation process offers tangible benefits. As Ben Sexton of JND eDiscovery notes, early validation ensures that legal teams can proceed or recalibrate before significant time and resources are spent. 

Additionally, validation builds trust. By combining quantitative metrics with the qualitative assessments of AI rationales and citations, legal ops professionals can minimize the “black box” perception and reinforce client confidence. Small-scale validation also reduces the risk of missing critical documents or overproducing irrelevant material, improving efficiency across the review lifecycle.

Although generative AI is most effective when paired with human intelligence, statistical validation provides a defensible framework. When faced with complex review projects, validation helps identify divergences between AI outputs and human expertise. If validation occurs early in the process, adjustments can be made to the prompt criteria, thereby achieving the desired results.