Choosing AI Tools
There are a variety of generative AI tools available and some of these are far more suitable to use in legal practices than others. Lawyers and law firms need to conduct due diligence when making decisions about acquiring or using such tools. They should be prepared to do that diligence by having a list of questions or factors to address, whether these are questions relating to confidentiality, data privacy and security, transparency, potential bias or any of the topics listed above. Does the company you are contemplating using adhere to certain principles in the delivery of its product?:
When considering the use of LLMs in your legal practice, keep in mind that many common sources of data are not fact-checked or reviewed for accuracy:
LLMs are often developed using internet-based data containing billions of words. However, common sources of this data, like Reddit and Wikipedia, lack sufficient mechanisms for checking accuracy, fairness, or appropriateness. Consider which perspectives are represented on these sites and which are left out...
If you instead build an LLM around more-carefully vetted sources, you reduce the risk of inappropriate or harmful responses. Bender and colleagues recommend curating training datasets “through a thoughtful process of deciding what to put in, rather than aiming solely for scale and trying haphazardly to weed out…‘dangerous’, ‘unintelligible’, or ‘otherwise bad’ [data].” While this might take more time and resources, it exemplifies the adage that an ounce of prevention is worth a pound of cure.1
There may be value in considering customized AI products for your practice, which can help mitigate the risks associated with the use of unreliable or inaccurate data sources.
In addition to conducting a careful review of any existing terms and conditions associated with use of a particular generative AI tool, lawyers and firms should ensure that they scrutinize their relationships with third-party AI providers – setting forth contractual provisions in procurement that ensure compliance with legal and professional obligations and allow for regular review. Contractual provisions setting out clear and strong data privacy and security measures are also pivotal to the use of these tools in legal practice, ensuring that confidentiality is always maintained.
It is important to be proactive, considering possible issues in advance to address potential problems instead of being reactive, looking for solutions after an issue has arisen that may attract liability or other consequences such as discipline.
Do some additional research on procurement best practices and recommendations relating to use. For example, in the United Kingdom, the Law Society created a comprehensive document on generative AI2 and certain sections contain recommendations specifically relating to procurement considerations such as:
- If considering the procurement of an AI tool, clarity on who will own the intellectual property rights over input and output data is essential.
- It is important to review the terms of service and privacy policy of each tool you use to check the usage rights, ownership of outputs and any disclaimers.
- If you are procuring or working with a vendor to develop a personalised generative AI product for internal use contained solely within your firm’s legal environment, you may wish to consider if and how you want to put confidential data into the tool, subject to the terms of use.
Some resources are tailored to inhouse counsel considerations, but many considerations would apply to private law firm counsel. 3
Key Takeaway: When considering what generative AI tool to purchase and/or use, lawyers and firms should consider factors such as accuracy and reliability, data privacy and security, customization, ease of use, integration with existing systems and customer support. When considering acquiring third-party AI tools, they should review any existing terms and conditions but also the advisability of contractual provisions that ensure compliance with legal and professional obligations.
Key Takeaway: Consider building an LLM around carefully-vetted sources and consider the benefits of a customized approach.
Key Takeaway: Conduct research relating to procurement considerations to assist with the required due diligence relating to generative AI use.