In part three of Law Carver's Cyber podcast series, we revisit post Julian Morrow's chat with SJ Price, leader of AI practice and partner of Sterling Rose about the implications of generative AI models for law practices. Welcome to Risk on Air. I'm Julian Morrow and today we're chatting about chat JPT and AI in the law. I'm joined by a living, breathing, human being and lawyer who's built a career around the dynamic intersection of law and emerging technology.
SJ Price is a partner at Sterling Rose, which I see from its website is a law firm and Emerging Technology Policy Institute. SJ, welcome. Thank you, Julian. And thanks for joining us to talk.
Everyone is chatting about chat JPT and using the letters AI, but what really is AI and especially what is it for lawyers to think about? So that's a great question because even the experts don't agree on what AI is. So we all have our own little version of what artificial intelligence is for me. It's when you have a machine that is doing something that typically or previously required human intelligence.
So for example, in the olden days last millennium, I don't know if you remember this, but we used to use maps that's called maps to actually try and find where to go. Now we just use our phone. So that for me is artificial intelligence. Chat GPT is a very interesting type of artificial intelligence and it is what is called machine learning.
And what is really fascinating about machine learning is that you don't have a human writing the computer code that creates a system. What you have is you have what's called a learning algorithm. That learning algorithm identifies very deep patterns in that information and uses those patterns or statistical models to create the AI system. And that's exactly what chat GPT is.
In fact, chat GPT has been trained on so much data that Julian, if you and I were reading together, we were both reading for 24 hours a day, it would take us a thousand years, both of us, to actually read all the training data. So if I type in a legally question into chat GPT, what's actually happening when I see that answer generated in front of me? First of all, what's not happening is chat GPT is not going off and searching for the answer. If you use Bing, that's a little bit different.
But chat GPT is actually predicting the mixed, most likely word in response to your query. So it's actually creating or generating new content in response to your query. It's just a statistical model that has learnt so much about the patterns in our language that is able to predict really well. So in terms of legal practice, under what types of circumstances could something like chat GPT be used, I suppose, what ways is AI already being used in legal practice?
It's actually been used for a really long time, particularly in discovery. Artificial intelligence is great at going through lots of information and just pulling out things that are relevant. Now, I don't know if you remember this, Julian, but last millennium, when I was a junior lawyer, I remember doing discoveries where you were going to enormous rooms with boxes packed to the ceiling and you would take a box down, take the lid off and physically have to go through the documents and read them. That's not the case now.
It's all done digitally. So lawyers have been using AI for a really long time. What is new, though, is the ability to use something like chat GPT to actually generate what would look very much like legal advice rather than legal services. And that creates a whole new category of risks.
What's at the top of your list for the risks that come with using generative AI in the context of legal practice? Julian, I got a top four list. Fantastic. Well, let's start with number one.
Okay. So this is the number one top risk hallucination. This is where chat GPT creates very authentic and authoritative sounding content that is utterly wrong. And sometimes it's not so subtly wrong.
And we've all heard the case of the American lawyer who relied in court on a list of precedents that have been generated by chat GPT that sounded very impressive. The only problem was that none of the cases exist. Exactly. Everyone's talking about the lawyer that used chat GPT.
And we all live in fear that there's a little bit of Mr. Schwartz in all of us. And I think there is. And that's why it's my number one risk, because you have very clever people like lawyers who are seduced by automation bias.
Sometimes it's not subtly wrong. It's just wrong. For example, someone used it to ask about the record for walking across the English channel and it proceeded to tell you who held the record as someone else got it to generate reasoning as to why mayonnaise is racist. So it is capable of generating data that's not even subtly wrong.
It's just wrong. But it's the subtle nuances of our language that are very dangerous. Lawyers really need to check very carefully the airport that comes from chat GPT in two ways. Number one, check the content with reliable sources.
But number two, really read the language carefully, because at a first reading, it can sound really good. But if you start reading it with a better and more detailed eye, you can actually identify some of the subtleties that is just simply not correct to give an example. Some of my friends are doing medical research and they've asked me to use chat GPT to generate abstracts. Now, the abstracts to me sound very good, right?
But they actually say to me, well, it is useful, but there's subtle points that I need to tweak to make it correct. That's why hallucinations is number one on my top four list. OK, well, so hallucinations will lock that in at number one. It's interesting, though, that the sort of strategies you describe to minimize the risk of a hallucination getting through without being picked, they sounded quite old-fashioned and traditional, checking sources and reading carefully.
They're two things that lawyers should be pretty familiar with in other forms. And it's just a skill that we're going to apply in a new technological course. Exactly. And you know what we are familiar with it, because hundreds of years, lawyers have had juniors and those juniors have done the first draft.
And then their supervisors have gone through those drafts. That's a really good way to look at chat GPT, treat it absolutely with the skepticism, the healthy skepticism that you review a graduate lawyer's work. Maybe even treat it with a little bit more respect than the way you treat a graduate lawyer. Dare I say it.
All right, so we've got three number one, but there are three more big items on at the risk list. What else is there? So my number two risk is ensuring that lawyers maintain the duty of confidentiality. And here, perhaps I should say, we're talking here about the consumer version of chat GPT.
We're talking about the one where you've got it on your phone or your computer. You go to the web browser and you log in. We're not talking about a situation where your law firm or your organization has API'd or whitelisted chat GPT or another large language model within your organization. Confidentiality is really important because, of course, what we do when we use chat GPT is we write a prompt.
And in that prompt might tell you a lot about what's going on in your law firm. And that's something we need to be very careful of. We need to ensure that we never upload in our prompts anything that's confidential or sensitive or proprietary or any personal identifying information or any client confidential information. Can typing information into a non-human piece of technology like chat GPT?
Can you bridge confidentiality to a computer? That is a whole other podcast. Yeah. But what we do know is we do know that OpenAI, who is the maker and producer of chat GPT, actually has humans who look at these prompts.
And one of the reasons which is a completely rational reason, they look at the prompts to ascertain whether the users are complying with the service conditions and their guardrails and things that it shouldn't be used for. So we do know that humans are looking at it. So as long as you have a human looking at your prompt, there is the risk of a breach of confidentiality. So regardless of what you might think of the way the technology is working, it's always prudent to work on the assumption that because it's a third party service provider, you are opening up information to a human somewhere.
That's exactly right. And even if you try to anonymize it, be very mindful that even anonymized data can actually pinpoint people. For example, if I upload something about a 99 year old man living in Perth, that really narrows the field as to who that person is. Yeah, I'll add to that point as well, by the way.
I will indeed. So we've talked about confidentiality generally. What about the question of client legal privilege in the context of chat GPT or other generative ais? Julian, I'm so glad you asked that.
One of the things that I think is really important is to remind clients that if they upload your legal advice into chat GPT, they may well be taken into a wave privilege. And some law firms, including ours, have actually updated our engagement letters just to highlight that to our clients who might inadvertently not really think about the implications of uploading their legal advice into a tool such as this. Really interesting because obviously that's an action that a client could take inadvertently, but then there could be a professional obligation to, in the best interest of the client, warn them about that possibility. Exactly.
And to build on that in litigation, and I'm not in litigation, but if I was, I would actually be asking clients, have you talked about your problem with chat GPT? Because it occurs to me that uploading prompts to the extent that they're discoverable or can be subpoenaed might actually be evidence of admission. Gasp of breath taken in at the prospect. But better to be aware of it now.
All right. What's next on the list? So number three is IP, intellectual property. And there's three aspects of this that are pretty interesting.
And there's cases going on on a lot of these. So number one, you've probably seen the case against OpenAI, who is the maker of chat GPT, where people are alleging that the training data that was used for chat GPT, the training data that would take us thousands of years to read, was actually uploaded in breach of copyright. So that's your first issue. And there is an ongoing case with regard to that.
The second one is it is statistically possible, but very unlikely, that chat GPT might create something that breaches copyright that actually replicates the prior works of human authors. Very unlikely, but certainly it's possible. And then the third aspect is because you do very little in the production of the content from chat GPT, arguably you may not be able to claim copyright over those outputs. So they're the three issues relating to IP.
All of which makes sense. And again, very sensible to be aware of those when we've got those fingers hovering over the keyboard of chat GPT. And you'll need to just build on that about maybe we don't own the output of chat GPT. I wonder if that's why the New South Wales Bar Association and their guidelines for chat GPT have recommended that barristers keep records of their prompts and the output, essentially like a provenance log.
And I wonder if that's in case they need to demonstrate that they have put enough human content or creativity or import to claim copyright. That's really interesting. So I suppose that aspect is also keeping records as a lawyer of your use of assistive technology, which would come in handy for all sorts of reasons. Exactly.
Okay. So we are three quarters of the way through SJ's list of top risks to be aware of in AI. What's coming in at number four is. It's a little bit of an unusual but super important one.
And that is judgment. AI is not appropriate to be used in all circumstance. And an example of this was there was a very tragic shooting in the United States at a university and a leader of an adjacent university created using chat GPT. What I would say was an incredibly empathetic note of sympathy and, you know, call to embrace diversity and so on.
But people were outraged when they realized it had been written not by a human. It's a very tricky one, but always think about it is appropriate for me to be using AI in these circumstances. And to be on the good judgment, that's one aspect. But the other aspect is as low as we have a duty of diligence and to do our work in a way that is most efficient for our clients.
So I think in the future, there is going to be a question asked about whether we should, in fact, be using these tools responsibly as part of discharging our obligation of due diligence and efficiency for our client. Because if it can be done more efficiently using the assistive technology, there's going to be a lower bill and provided the appropriate judgment and oversight has been applied, then maybe that is a better result for the client. Exactly. And I think it's important because what I'm seeing in legal practice is I'm using chat GPT to generate term sheets, which they bring to you.
They're using it to generate contracts, which they bring to you. They're using it to generate instructions to you. So our clients are already using it. And I've even spoken to general counsel of organizations asking them, well, you know, if a lawyer is using this tool, do you want to know?
Do you feel that they should be upfront and tell you? And he laughed and said, well, actually, I want to know if they're not using it. So I thought that was a really interesting juxtaposition in his view that there is an expectation that lawyers use all appropriate tools in a responsible way to deliver their legal advice to the clients. And that brings us, I think to the next question.
Talk about the things that individual lawyers should be aware of when engaging with the technology. What about firms? What measures should be put in place by a legal organization to ensure that the individual users are employing SJ's top four tips? And I suppose to ensure that there are records and processes in place to be able to refer to and rely on if things don't quite go as well as we might have.
That's exactly right, Julian. Two top tips, education and governance. And this is incredibly important because research is showing that there's 30 to 43% of people in organizations are using chat GPT to do their work to enhance their productivity. And of those, 68% are not telling anyone that there's this shadow use.
So my first tip would be address it, tell your staff how you want them to use it or if you don't want to use it. I think that's really important. A lot of organizations have called a pause, have said we don't want anyone using this forward until we figure out how we're going to embrace it, how we're going to manage it and govern it. The next thing is there's tremendous value in educating people, educating and consulting.
So educating people in your law, things about what artificial intelligence is, what machine learning is, how it can be used and what it does and engaging in the discussion. And there's lots of stakeholders, there's the lawyers in your film, but there's also your clients and other staff that also have opportunities to improve productivity in a safe and responsible way. The second thing is once we have this education, then it enables sensible decisions to be made about how we govern this technology. For example, coming out with some sort of philosophy about how you will use it or principles about how you will use it and a strategy because there's tremendous opportunities for organizations that are prepared to embrace this, to be competitive, to deliver a more personalized service for their clients.
And then what we're seeing is a lot of organizations are actually having a policy about how their staff can use or chat GPT. And that policy is essentially, we encourage you to experiment and try this, but don't upload any confidential personal proprietary information. Use good judgment. And let us know if you're using this because we want to make sure that it is being used in an appropriate way.
I suppose if you ask that GPT to generate you an AI policy in your law firm, you should probably get that looked over, be wary of hallucination and all the other tips as well. Exactly, Julian. The chat GPT is one aspect of technology related concerns that really the focus of lawyers and others' attention at the moment. The other, of course, is cybersecurity.
How do AI and cybersecurity fit into each other? Are they separate buckets or do they need to be looked at together? They need to be looked at in a consolidated fashion. And the reason I say that is if you're using artificial intelligence and you're uploading data, you want to make sure that that data is safe.
The second thing, and I don't know if you've noticed this, Julian, but I am getting really sophisticated phishing emails. It's really changing. And my view is that is from Sam is using chat GPT. It is now I'm getting things that if I don't look carefully, it looks like it comes from someone at my law firm, Sterling and Rose.
And it's asking me to do very fun things like get my units for my employee share scheme and all sorts of really quite enticing opportunities. So there are many risks to attend to, but I suppose we should also stay tuned to the fact that there's immense potential here as well. Could you give us a couple of specific examples that in your mind sort of point to some of the efficient or creative or just interesting ways that AI technology can be applied in the field of law? Absolutely.
And the first one fact is that GPT-4, which is the next iteration from chat GPT, is able to pass the US uniform bar exam in the 90th percentile. It can also pass the LSAT, which is the examination to get into law school in the 88th percentile. And this is important because you only need to pass it in the 90th percentile to be considered for Harvard Law School. So GPT-4, almost going to Harvard Law School.
Now more fun is some of the things we're using it for in real life in the wild. Here's some examples to help you get really cool and fun titles for articles or presentations. An area where lawyers could potentially improve. I'll give you an example.
We had to do a presentation, I and some of my legal colleagues, to an international audience. So what we wanted is we wanted to give it an Australian flavour, right, to lure people in with the whole Aussie vibe. I used GPT and what it came up with was Sizzle or Fizzle, cooking up innovation down under. But that was great, right?
And we had this whole barbecue thing happening. So that's one example. So if you have to write an email or a message to someone and you want it to have a particular time, for example, if you have someone in your law, maybe who isn't keeping up with their time sheets. And what you really want to say is, look, dude, you need to keep up with your time sheets, right?
Well, chat GPT can actually make that sound very polite and very nice. And we understand you're very busy, but this is really important and makes it a very nice style. For example, you can learn things. So if I had a client whose domain specialty was quantum computing, I could ask chat GPT to explain to me, well, what is quantum computing?
And it might say something I say, oh, that's too complicated for me. Make it simpler for me. And so you can use it as a personal tutor to enable you to understand new domain areas. Julian, one of the other things that chat GPT is incredibly useful for is to check your spelling and your grammar.
So there are no excuses anymore for having bad spelling and bad grammar. And this is particularly important if English is your second language. You can use this as a tremendous English graduate to check your work. Fantastic.
Chat GPT is also great to summarize notes. For example, when we get co-pilot, Microsoft co-pilot, it will actually take a transcript of a meeting that you're having, say, on Teams. And then it can summarize the meeting and identify the action items who has to do them in by what time. This is an incredible productivity boost.
Now, for the one I hear lots of lawyers using it for what we've discussed, which is to do your first draft, your first draft of a particular course, your first draft of a contract. But the uses are only limited by the imagination of lawyers who are incredibly clever people. So watch this space. And I suppose it's worth bearing in mind also that chat GPT is one large language model and generative AI.
There are others out there as well. And I imagine that we'll see AI integrating into all sorts of apps and technologies that we're already using. It might become part of our basic word processing software. It already has, I suppose.
So there really are going to be lots of both general work and law specific options becoming available and worthy of exploration. That's exactly right, Julie. And you know what? It's closer than we think.
Because those of us who are in the Microsoft 0365 environment, if you're in the United States it's there now. But shortly, watch this space. It's coming to Australia. It's called Microsoft 0365 co-pilot.
And what it is, it's GPT4 within your desktop environment. And what this enables you to do is to generate content by pointing it to the documents within your organization that you have access to. So that means you can say, look at this word document and create me a PowerPoint. Or look at this word document and put that information in a tabular form for me.
Interesting stuff. It sounds like the sort of thing that in the past you might have been getting junior lawyers to do. And I'm sure that's not going to lead to junior lawyer redundancy. It just means that the humans are going to be doing more high level and more interesting stuff and the computers are going to take the ground down.
Exactly. In fact, it's a great time to be a lawyer because you're going to bypass all those boring, boring tasks that lawyers like us have to do when we first join the workforce. Well, it's been great chatting at GPT with you and lots to think about, but also lots of benefits along the way as well. So I suppose it's good luck in the Wild West of New Technology.
Thank you, Julian. Thanks for listening to Risk on Air by Law Cover. Join us for the next episode on current risks and legal practice to stay up to date.