Use is growing and the benefits are numerous and significant, but what do law firms and lawyers need to understand about AI tools like predictive coding and technology-assisted review?
By Paul Hunter and Phil Smith, FTI Consulting
Predictive coding and technology-assisted review (TAR) have sparked considerable discussion among e-discovery practitioners and lawyers around the world. These discussions have been taking place for years in the U.S., but the technology has just started to gain meaningful ground in the UK, and more recently in Australia. While adoption in Australia is quickly evolving, the industry continues to question and debate the extent to which predictive coding results can be defended and whether it is truly a reliable method for more effectively leveraging the effort of human reviewers in e-discovery.
Machine-learning technologies like predictive coding bring many benefits to the legal industry – particularly the ability to reduce datasets quickly, find important information early on and save considerable money on document review. With document sets growing exponentially, lawyers are struggling to navigate the data in an efficient and defensible manner. Predictive coding can automate many of the time-intensive manual processes involved with keyword search, filtering and data sampling to prioritise likely responsive documents and can usually dramatically reduce the number of non-responsive documents that need to be manually reviewed. But the technology is often considered to be a black box, lacking transparency into how results are obtained.
At the same time, many lawyers understand that the existing human processes are imperfect, and often result in inconsistent responsiveness coding. Predictive coding can provide equal or even improved precision when compared to manual methods or to keywords which have not gone through a sample-based refinement process, and the technology can be relied upon for consistency in making the same call time after time, based on what has been learned from the set of human-reviewed training documents. This ability to automate much of the document review process increases efficiency and can improve accuracy and quality control. The key is for lawyers to be careful about selecting tools that genuinely allow control over the processes and visibility into coding decisions.
Understanding the benefits, defensibility and accuracy as compared to human review, and testing to see how effective various methods including predictive coding will be on the particular matter, are important first steps towards adoption that many legal professionals in Australia are beginning to take. Beyond the initial acceptance of the technology as a viable option, there are additional considerations lawyers should keep in mind as they begin using predictive coding. These include:
As this trend continues, it is crucial for in-house and outside counsel to understand the nuances of applying predictive coding and how to determine which type of cases will benefit most. They must also be prepared to address the use of predictive coding early on in a case and clearly define parameters in conjunction with opposing counsel and the courts, to ensure that all parties are on the same page with its use.
Paul Hunter is a principal research scientist at FTI Consulting’s Technology practice in Australia.
Phil Smith is a director at FTI Consulting’s Technology practice in Australia.
Related stories:
Court rules in favour of predictive coding
In the face of adversity: Law firms and their digital future
Predictive coding and technology-assisted review (TAR) have sparked considerable discussion among e-discovery practitioners and lawyers around the world. These discussions have been taking place for years in the U.S., but the technology has just started to gain meaningful ground in the UK, and more recently in Australia. While adoption in Australia is quickly evolving, the industry continues to question and debate the extent to which predictive coding results can be defended and whether it is truly a reliable method for more effectively leveraging the effort of human reviewers in e-discovery.
Machine-learning technologies like predictive coding bring many benefits to the legal industry – particularly the ability to reduce datasets quickly, find important information early on and save considerable money on document review. With document sets growing exponentially, lawyers are struggling to navigate the data in an efficient and defensible manner. Predictive coding can automate many of the time-intensive manual processes involved with keyword search, filtering and data sampling to prioritise likely responsive documents and can usually dramatically reduce the number of non-responsive documents that need to be manually reviewed. But the technology is often considered to be a black box, lacking transparency into how results are obtained.
At the same time, many lawyers understand that the existing human processes are imperfect, and often result in inconsistent responsiveness coding. Predictive coding can provide equal or even improved precision when compared to manual methods or to keywords which have not gone through a sample-based refinement process, and the technology can be relied upon for consistency in making the same call time after time, based on what has been learned from the set of human-reviewed training documents. This ability to automate much of the document review process increases efficiency and can improve accuracy and quality control. The key is for lawyers to be careful about selecting tools that genuinely allow control over the processes and visibility into coding decisions.
Understanding the benefits, defensibility and accuracy as compared to human review, and testing to see how effective various methods including predictive coding will be on the particular matter, are important first steps towards adoption that many legal professionals in Australia are beginning to take. Beyond the initial acceptance of the technology as a viable option, there are additional considerations lawyers should keep in mind as they begin using predictive coding. These include:
- Flexible to scope and size: There is a common misconception that predictive coding is beneficial only for large, complex matters. But in reality, the technology is applicable across the spectrum, as long as it is implemented the right way. We’ve seen many small cases where predictive coding was exactly what was needed to get the job done quickly and cost effectively. As an example, one client that had not previously used any form of TAR was looking for a fast way to find the key facts in a case. There were only 50,000 documents, but the timelines were tight and it was an extremely small team dedicated to the case. Using a Continuous Active Learning (CAL) approach, our client was able to review a small sample set, and ultimately find a high percentage of relevant material in a very short timeframe, reviewing less than half of the document population.
For matters that involve a high volume of responsive documents, it often makes more sense to implement full scope predictive coding, rather than just CAL. Likewise, for cases that involve a mix of hard copy documents and electronic documents, it is better to deal with the hard copy documents manually, and use some form of predictive coding on the electronic documents to balance the manual work with a highly efficient approach for the digital portion. These variables are reminders that predictive coding can be flexible for many different matters, and decisions about which approach to take should be based on a case’s unique needs.
- Balancing with people and process: It is important to view predictive coding technology as one tool in a larger workflow that also include visual analytics and data mining, people and established processes. Every predictive coding matter should involve experts who offer legal, technical and statistical expertise, can guide the team through the process and help the legal team become comfortable with the technology. Involving an expert third party can assist with decision-making about which issues in the case would benefit from predictive coding, assigning the number of documents to review in the training set and designing iterative workflows. The right people working with the technology through transparent processes will also help demonstrate defensibility of the results, as can the use of data mining to substantively understand the results. This becomes increasingly important as the size and complexity of a matter scales.
- Court rulings: We have seen recent rulings in the Victorian Supreme and Federal courts that have shown judicial acceptance of the use of TAR. Although both of these matters have specific requirements as to how the technology should be applied, they offer some initial guidelines for legal teams looking to adopt a similar approach. Lawyers should be paying close attention to current and future judicial rulings on predictive coding, and begin to understand how it works, so they are able to defend its use in courts. Landmark rulings around predictive coding in the U.S. and the UK are also demonstrating to the Australian market how the technology is being used and approved on the global stage.
As this trend continues, it is crucial for in-house and outside counsel to understand the nuances of applying predictive coding and how to determine which type of cases will benefit most. They must also be prepared to address the use of predictive coding early on in a case and clearly define parameters in conjunction with opposing counsel and the courts, to ensure that all parties are on the same page with its use.
Paul Hunter is a principal research scientist at FTI Consulting’s Technology practice in Australia.
Phil Smith is a director at FTI Consulting’s Technology practice in Australia.
Related stories:
Court rules in favour of predictive coding
In the face of adversity: Law firms and their digital future
