I have a request of you. Last week, I described AI use recommendations of the California State Bar. Those recommendations might be a good foundation for internal law firm AI protocols. Similarly, last week Justin Daniels (Baker Donaldson) focused the attention of the AI PG monthly call on the NIST AI RSM framework. (Justin’s excellent slides are attached.) NIST’s framework is the best we have right now for any company use. It seems that we’re all searching for guidelines for the incorporation of AI into businesses. Not unreasonably, as Salesforce reports that 28% of workers are currently using generative AI at work, and half of them without formal approval.
So, are you pleased with your law firm’s AI policies? Please share them with me in an email (with your firm’s approval), along with your thoughts.
At the same time that we install AI guidelines we need to look at the fundamentals of what we do. Here are a few choice quotes from Jordan Furlong’s most recent opinion on ‘quality control’ as the legal skill needed for the next few years of the legal profession:
“We simply don’t know how Generative AI is going to be used for legal work even a few years from now. ... The pace of change in this area is too fast and the direction is too unpredictable.”
“[Additionally, up to now] the legal industry has not fostered a culture of standard work, error detection, peer review, performance measurement, and continuous improvement. Likewise, law does not demand evidence-based, data-driven practice. The legal industry does not rigorously assess the efficacy, quality, and value of legal services.”
“Maybe the core competence that we [ought to] educate, train, and evaluate in lawyers is the ability to assess the quality and effectiveness of legal products and services and determine whether they’re fit for purpose — regardless of whether they were generated by machines, or people, or both.”
“[For example] give a [bar] candidate the facts of a client matter, and assess them on how effectively and efficiently they can instruct AI to produce an analytical and advisory memo, and how accurately they can then critique and improve the memo the AI produced.”
“Let’s combine these two developments [Generative AI and new lawyer competencies] and bring the legal profession to a level of sophistication and performance it’s never seen before.”