I'm Mary Ann Kolbasakmke, Executive Editor at Information Security Media Group. Today, I'm speaking with Damien Chang, who is Business Information Security Officer at SASE Bender, Netscope. Damien is a former Senior Director of Cybersecurity Engineering at Healthcare Provider Organization Dignity Health. We're going to be discussing generative AI and healthcare.
So, Damien, based on what you see, how is generative AI being used in healthcare mostly right now, and what are the potential hotspots for its use in healthcare? I think generative AI in healthcare is a tricky subject, and simply because we're having to take into account patient privacy. And so, I've seen many use cases pop up on the internet, and I kind of wonder if that's really allowed within our industry. For example, generating patient care plans or follow-up letters, and I think a lot of those use cases will help out a lot of our clinicians and doctors become more efficient, but we really have to think about what type of data we're putting into public AI systems like chat GBT.
How much of that information is really going to stay private versus be used potentially against patients later in the future? And so, I think although generative AI is becoming a hot topic, it hasn't really taken off as much as it has more recently becoming in the news more often, and users trying to find more ways to get around the aspect of enhancing their day-to-day jobs. But I'd be really concerned about what types of data we are feeding into these models, and can we still maintain patient privacy? So, when it comes to those concerns related to patient privacy, what are you most worried about?
Is it patient data getting fed into these systems and not being properly de-identified? What are your top concerns? Yeah, I mean, so if you are a user who is trying to build a specific patient care plan, what information are you putting in there about that specific patient? And I think we need to really slow down and think about the proper ways to use these AI models.
And one of the things that have come up in my conversations has been the use of potentially moving these AI models privately, internally to health organizations, where now we can use those models in a secure fashion and also use it against our own data. But I don't think we're there yet. I don't think we're able to pull those models into a private instance and just leverage our own private data. I think we need to architect a way for the industry to get there.
But right now, we're just not there. And I would be worried about putting that information out into a public repository where that data could later be used against patients or used against our organization. We also hear a lot about generative AI sort of being good at taking a lot of data or a lot of information and presenting it in a way that sounds good. But then when you start looking at it, there's a lot of inaccuracies sometimes.
And I guess when it comes to patient health, you don't want bad information to show up that a clinician might act on in a way not knowing that this might not be accurate. What are your concerns when it comes to the accuracy that these systems may or may not have? I mean, that's a very good question. I kind of compare it to let's turn it around.
Patients who are trying to find their own cures. And remember when websites such as WebMD popped up, people were just searching for their conditions and trying to come up with a solution for their own conditions. And that's sort of happening with AI today where a patient will go and try to self-assess and maybe delay that real treatment that they need going forward. But you bring up a good point.
How do we make sure that these AI models are being trained correctly? And how do we know that the sources they're using are legitimate? And at this point, we really don't. I think one of my concerns, especially if users are starting to use a generative AI system to, let's say, do some sort of research on their behalf, how do I know that source isn't poisoned?
In healthcare, this is very easy. What if I go and search for conditions and the AI tells me that, oh, don't worry, that's just allergies. And so you never get it addressed. And I look at that as a way of attacking an entire population.
If I were trying to not have people worry about their condition and maybe seek a professional. But if I can downplay those symptoms by training the AI model to just ignore it or tell the population, it's just allergies. You don't have to worry about it. Meanwhile, this could be a buyer or a pair attack or some sort of new virus that's spreading across the world.
I think within technology and even cybersecurity, that's also a fear of ours. Where is that source coming from? Who's been training it? And are we susceptible to an attack?
But yeah, that is a very good point. I don't know if we were able to address that today. So now it didn't mean I understand that you have some concerns or some issues or some thoughts about how generative AI might force healthcare systems to rethink and overhaul their supply chain security strategies. Why and how might that happen?
The supply chain is always at top of mind when we're thinking about technology and security. Because now that AI is entering in as becoming more mainstream, we're going to be forced with multiple vendors that claim they have AI. And as a CISO, here are the things that I be worried about. One, is it really AI?
And two, how do I know that each of these tools, their AI systems won't counteract with each other? And to what extent is that AI really learning from your data or potentially using that data to train other models for other customers? So I think this brings into another piece of third-party risk and vendor management that we're going to have to take a look at. We're not going to be able to really dig down deep into every single tool.
But as a technologist, I'm now looking at my users who want to use AI, but also my vendors who also want to use AI, and then use it as a selling point for their platform. How do I make sure that that is going to benefit me and my organization and my department from becoming, helping them become more efficient? But how do I also make sure that they're not kind of racking with each other and canceling each other out? And that's what I'd be worried about when it comes to supply chain, the third-party risk, and how that's going to impact our organizations and healthcare providers in the future.
So Damien, with that said, how should healthcare assess generative AI when it comes to their suppliers? Are there certain questions they should be asking? Are there certain red flags they should be looking for? And how should they sort of move forward on that path?
Yeah, I think it starts with being aware of who your vendors are and what they're using to really provide services for you. And so they're using generative AI to create something. Do they even have the rights to use it? And would they be protected from copyright infringement?
And so a lot of these are concerns of mine being on the vendor side so that we're not taking code and utilizing code from an AI system because I just don't know who's created that and who may come back to claim it later. That's a big worry of mine. So if you're going to assess vendors moving forward, ask them if they're using any kind of AI within their platform to deliver services to you. If they're using any kind of generative AI to create content or create source code, you need to understand that aspect of it as well so that you can be prepared and ask the right questions.
It's, you know, go back into schools and students using generative AI to produce their papers. We also now have to look at our vendors to see, well, how much generative AI are they using to support their business? And does that create an increased risk to you as a potential customer? So Damien, as you know, we hear a lot about software bills of material where, you know, vendor provides or will hopefully provide their customers with a list of, you know, the various components that are in their products in case there is a vulnerability found somewhere that, you know, is a commonly used open source component or something that's, you know, popular and used in many different sorts of applications.
Do you think we might get to the point where there may be a need for a software bill of materials when it comes to the AI that's used in the technologies that are sold to healthcare companies and providers and other sorts of users that will depend on these technologies, but yes, they want to know where this AI is coming from. Right. Where it had originated from. I think that might be a good idea.
I'm just not sure how many organizations would actually put that in the S-bomb. We, you know, I think as a full disclosure, right, you'd want the providers to give you that information. So maybe that is the right place to start. I'm just not sure.
I think as an industry vertical, we're going to have to come together and create a standard where it's openly talked about and we understand where some of these services are being supported from. And finally, Damien, we talked a little bit about some of the privacy issues, some of the patient privacy issues that involve the use of generative AI and healthcare and potential issues with that. What about security issues? Do you foresee there being certain security red flags when it comes to the use of generative AI in healthcare and where would those issues perhaps be focused, do you think?
Yeah, I mean, we want to allow our users to explore new technologies. And I think within healthcare, we don't want to stop innovation. And if we just put a wall up and say, no more generative AI, users are just going to find a way around that. And then as a security team or a technology team, you lose the visibility to what those users are doing and you lose visibility to the risk.
I think one of the things I would be more concerned about as a security professional is not necessarily that the users are going there, but that hopefully they understand the proper uses for generative AI and what's acceptable. And maybe put some guardrails around that where we can have DLP rules in place that stop them from uploading any type of PI or patient information into these systems that we're just not sure how they're using that data into other models that maybe use for other customers. And so my approach as a security professional is to not really block the users but educate them and then put guardrails around them so that as soon as they do something that looks a little risky, we can then put a coaching page or coach them in real time or perhaps block that specific action but not cut out generative AI tools altogether. I think they're going to be useful in our vertical and for user workflow and it's going to make them more efficient.
And we want to be innovative like I said and not stop these users from trying out new tools. But again, as a security professional, I'm looking at risk, right? What are we going to gain versus potentially lose with these types of tools? And I think there's a lot to gain, but there is also potential to lose a lot.
We just have to be sure that or be understand what those users are trying to do and put guardrails around the data and patient privacy. Well, thank you so much, Damien. I've been speaking to Damien Chang. I'm Mary-Ann Kobusak-McGhee of Information Security Media Group.
Thanks for joining us.