Welcome to Cybersecurity Insights, the podcast for the CyberEd.io learning community. Our goal is to bring Cybersecurity practitioners, the latest and most relevant education and training to upskill and dive deeper into topics that matter in today's modern Cybersecurity world. Good day, everyone. It's Steve King.
I'm the managing director here at CyberEd.io and on today's podcast I have Oliver Tevakoli, who's the Chief Technology Officer at VectorAI, which is a AI-driven threat detection company. Prior to joining Vector, Oliver spent more than seven years at Juniper as Chief Technical Officer for the security business and also co-founded Trilogy. Prior to that, he did stints at Novelle Fluent Machines and IBM. He received a Master's degree in Mathematics and Undergraduate in Mathematics and Computer Science from the University of Tennessee.
So welcome, Oliver. I'm glad you could join us today. Thank you, Steve. Glad to be here.
Terrific. Let's just jump in here. I've got some questions. We know we've got increased threat volume and lots of products and layers and tools and role resulting in more alert volumes.
Today's socket analysts are overwhelmed with alert fatigue and along with CISOs or suffering from burnout as well. Sounds like you guys, VectorAI solved some of that. Can you give us an audience and explanation on how you do that? Sure.
I think we recognized early on in the formation of the company that noise is a problem. Noise, overwhelm is anything positive you might do. And so we started from day one being maniacal about trying to extract as pure a signal as we could out of activity that is observed against different attack surfaces that a customer might have. But then we realized even that really wasn't fundamentally enough.
And what we needed to do is to aggregate that signal in a sense that if you see one particular signal in isolation, that could be good. It could be bad. If you see it a thousand times, that same signal, it could be bad. But the way you need to look at these problems is what is the combination of signals that are related to each other?
What story do those tell in helping you figure out whether something could do you agree with someone or not? And so for us, I think from day one, we felt that as I refer to it, you can apply fancy math to these problems and get pure signal. You can use fancy math to do better aggregation and better prioritization. And so whether that's AI, whether that's ML, whatever techniques we might apply, start with assuming this is a noise problem and tackle things from that direction.
Yeah, that makes sense. When you talk about signals, what signals in particular are you talking about? I mean, the signals can be something as simple as, you know, a relatively rare act. Something that happens quite infrequently and it may not in and of itself be malicious, but if it were to happen that the security team would probably want to know about it, it can be, say, excessive failures on MFA log-ins, right?
It's like, yeah, that may just be a user-passed bad thing during their code multiple times, or it could be an attacker coming in. It shouldn't happen that often. And so you minimally want that signal to be surfaced. On the other hand, you may have much more sophisticated signals like communications that a asset that you have inside your enterprise is performing with some entity outside the enterprise.
And planning back and swinting at that and judging whether it might harbor a command and control channel or not is a much harder problem. So it's not about this one thing happening, this one unusual thing happening, but it's rather something that's cloaked in the clothing of something that is relatively typical in the enterprise with being able to discern usage patterns that are likely indicative of command and control. And so for something like that, we use deep learning and neural nets to actually try and separate the good from the bad. How do they manifest themselves, I guess, is what I'm asking.
Are these alerts that come from a SIM, for example? Yeah, I think, yeah, for us, we oftentimes feed information into a SIM. But basically, we observe things that are actually happening in the environment. And that might manifest itself through traffic that we see on the network.
So we might have a means of tapping the network, extracting the traffic, extracting metadata out of that traffic, and then processing that metadata through a series of algorithms that look for different kinds of nefarious behavior. Or it may be that we are pulling logs of activity in systems. So if you may have, for instance, your identity system that you federate through something like Azure AD, which is, I guess, now called EntraID as of last week, from Microsoft is a relatively common use of it. It's like, you can tap into the logs that describe activity in that system.
And then, eventually, you have to understand back and say, which of this activity looks potentially malicious? And so what we're trying to do is take raw information of activity and produce out of that in a much more concentrated and relatively noise-free signal of things that could be potentially bad. So that's like net flow data, for example. I mean, net flow data is an example of net work data.
We actually tend to go deeper than that. So net flow data typically just says these two entities have a relationship with each other and they're transacting this amount of data over this point in time. But if you look in most corporate networks, you have traffic using a variety of protocols, even if it's encrypted and you're using TLS, there are certain elements that are visible on the handshake before the encryption kicks in. If it's internal and it's using authentication protocols like Kerberos or using file sharing protocols like SMB or remote procedure call capabilities like MSRPC, all of those things are visible on the wire.
You have access to the raw packets. And so what we typically do is get a hold of the raw packets, extract the metadata necessary to go prosecute the mission that we have, which is to find signals. And so net flow oftentimes is insufficient for that. You need to know exactly what RPC was executed and try and divide Y, rather than simply knowing that there is conversation occurring between two machines on a given port, which is what net flow gives you.
So that implies that you actually do packet inspection then? Correct. I mean, certainly for our, again, we have two distinct approaches that we take in terms of, well, the approach is the same approach. We try and get raw data and divide, you know, grab signal out of it.
But that raw data may either be network traffic, in which case we get a hold of the raw packets passively and extract the necessary data out of them. Or it may be systems of record out of which they're producing logs and activity. There can be public clouds like AWS where you're used to kind of seeing things like in CloudTrail. It can be in Azure.
We use the Azure logs. It can be in GCP. It can be in identity systems such as Azure AD or now EnterID. It can be Active Directory Domain Controllers on-prem.
So there are a number of systems that produce very valuable raw log activity. So we will pull that information and then ultimately look at it in aggregate and divide patterns out of it and learn what is normal, what's abnormal, what could potentially be harmful in the environment and then provide a relatively thin stream of actual alerts to the SIM or to the SOAR or to wherever the customer chooses to center their operational experience for their analysts. Right. You included AI in the name of your company.
I'm assuming intentionally. So how do you leverage AI and what particular AI do you leverage? Sure. Yeah.
To drive the engine. Yeah. AI is a term of art that gives us a pretty long time multiple decades. At any given moment in time, it has meant different things.
Right. In the 80s, if you ask people about AI, it was all about expert systems. We interviewed experts and then wrote code to capture the knowledge of those experts in systems. Nowadays, I think a lot of what AI means oftentimes is forms of neural networks.
These are what underpinned today's kind of revolution of generative AI, the algorithms, the large language models, the chatbots that you see out there. But those neural nets, which are also kind of the basis for what's referred to as deep learning, which are the basis for even five, ten years ago for Google Translate and other things like that. One more five years ago, not ten years ago. So neural networks and the capability of doing kind of recurrent neural nets and other stuff like our techniques that we have used extensively in service of trying to find the signal.
So our command and control detection as an example is constructed on a series of neural nets with LSTMs and recurrence and all these other kinds of things. That form of AI is substantially different than the generative form of AI. So if you think about it from a security perspective, what you want is you want to define good from bad, which is in the term of art, but that is discriminative AI rather than generative AI. Generative AI is, you want to ask questions or you want to give instruction and how the system creates something for you.
Either an answer or a picture or a deep fake in terms of voice or video or something like that. So for us, before this latest craze of AI, which is around language models, large language models, which really began probably in earnest maybe six or nine months ago, we had AI in our name because we fundamentally used a variety of techniques that in the common parlance fall under the banner of AI to solve discriminative problems, which is to tell good from bad. So you're, I assume, looking at behavioral patterns of single types then, kind of like if you see this and there's one of these after it, that probably implies that something is happening. Yeah, but that would be a more straightforward kind of analytic process, right?
I think what you try and do with a lot of the AI techniques these days is to say, we're going to give you training data. We don't quite know how to discriminate. Let's imagine you had all this communication that occurred, high volume communication that occurred from assets inside the enterprise to entities outside the enterprise. And so the question is, which of these persistent connections to the outside have indications of C2, of command and control?
And so largely, it's very hard for a human to pour over the time series data, the ebb and flow of communications between the internal entity and external entity and say, aha, I had an epiphany. Here is why this, I would call this a command and control channel, and here's why I would call this other thing, not one. At the extremes, yes, you can come up with that. But ultimately, what you want to do in this modern world and this modern age of AI is really build training sets, build examples of things that should be detected as command and control and things that ought not to be detected, and then train up a neural network to distinguish between those two.
And that's a, when you're done training up that neural network, I mean, today with LLM's, you tend to hear about these models that have 65 billion floating points or 175 billion floating points. When you're trying to do a more narrow task like a C2 channel detector, you can train up a more modest model, one that has maybe 10,000 floating point values. But basically, you train it up by running examples of positive and negative through it, and then you ultimately generalize that for it to be able to detect in the wild, new examples, new positive examples of command and control. And so it's not that you do the old fashioned thing of a human cogitating and saying that is the pattern that I'm looking for.
Instead, humans will produce very carefully the sample data sets, and then a neural network will be trained up to distinguish between the good and the bad. Ah, I see. Okay. That makes sense.
We've been kind of, I don't want to say talking about this for a long time, but there have been, as I'm sure you know, lots of companies that have been working on similar approaches. That's fair to say over the last, I mean, just we worked with, you know, my former life with a company out of Palo Alto SRI, I think it's called Tessera, which did that sort of that kind of packet analysis and try and had a predictive analytics component to it. The way that you is the question is, is the way that you use AI, is that what differentiates your approach from other folks that have done something similar in this space? Yeah.
I mean, I think at a certain altitude, everything starts to sound the same, right? It's like, oh, we apply, you know, we apply fancy math, the problem of finding bad stuff amongst a pile of, you know, not so bad stuff. I think the approach that we take is fundamentally different. We start with the, we begin with the end in mind.
A lot of times what you hear companies talk about is to say, it's about the unknown unknown. It's about the anomalies that you find, like, okay, let me find all the weird things that are occurring in this enterprise. And at the end of the day, if I sum up a cluster of weird things and they go over a certain threshold, I'm going to say you should go look at that. The problem with that approach is kind of what we call an anomaly-based approach, which is a clustering approach, which is an outlier approach.
Can I find the outliers? Is that humans, as I told them, carbon-based lifeforms tend to be rather unpredictable. And if you go into the average enterprise, there's a fair bit of entropy. Both in terms of what users do from day to day and what applications as they get upgraded, how they appear to behave from day to day.
And so an approach that focuses on the different, on what is different from one day to the next, tends to, we believe, at its core, end up being too noisy. And the way that noise is then managed in other systems is to simply threshold the problem, right? We're going to say, okay, it's got to be this weird, that we only generate, you know, 50 signals a day as opposed to 500 signals a day. And in doing so, in thresholding the problem, there's actually no guarantee that you haven't just thresholded in the problem to a degree where the attacker is now under threshold and not being triggered, but some weird users who just misbehaved in extreme ways are above threshold when getting alerted from.
So we fundamentally disbelieve in that as the core theory for how you approach this problem. So we start with security researchers in mind. We think of it as almost like sequencing the DNA of a task. That sequencing is unique to each attack surface.
So if you look at how would I attack AWS versus how would I attack Microsoft v65 versus how would I attack, you know, an on-prem network? Those are each different attack surfaces. They are like snowflakes. They share very little in common.
But so we set our security researchers to the task of saying, what are really the core principles for how you attack this kind of an environment? And then can we design basically trip wires, think of them as the alerting trip wires in various parts of that ecosystem, so that it would be very difficult for an attacker to actually work their way through that environment. Without setting off multiple trip wires. So by the time we apply math to the problem, by the time we bring our data sciences into the fold, we've already kind of figured out what signal we want to go search for.
And then we set ourselves a task of maniacally finding those different signals in service of each of those trip wires. Which is, as I said, very different than I'm just going to look for weird stuff. And if there's enough weird, then I will trigger and then give it to you. And so we believe our methodology is the right methodology.
It's born by the fact that, you know, customers who have, who ended up deploying our systems tend to save us. Very, very low-term rates. And in the security industry, that tends not to be the case. You tend to have a lot of companies that produce something that sounds great on paper.
And then, you know, a year later, the customer has moved on because it hasn't actually proved itself to be effective. So we think the methodology of having a sense of what bad you're looking for and then going and applying these mathematical techniques to find that bad and then chaining those bads together is the right methodology. Yeah, it sure sounds like it. I mean, much more effective than, you know, as you say, accepting all noisy sort of stupid chatter, a stupid, meaning behavioral activity that could represent weirdness within the human factor condition.
If you think about it, if you think about it, an advanced attacker tries to keep weirdness to a minimum because they know that too much weirdness will likely trigger things. And yet, you know, we've all had employees who have triggered every weird thing imaginable, who do things outside of policy, who do weird things. It's usually in most anomaly-based monitors. Oh, you know, this person just accessed the system locally from California.
And now they're accessing it from China. It's like, yeah, they happen to be in China and sometimes they're using the VPN. And so they're not using the VPN. And sometimes they look like they're in China.
Sometimes they look like they're local. And that's not that unusual. Like people traveling remotely and accessing systems within without VPN is not the really weird thing that we like to think it is, right? And if you trigger each time, that happens.
And you're sending your security teams on a goose chase. Yeah, yeah, sure. So I'll think of you. I think you either coined or used the right phrase in signal clarity.
So that's a non-generalized signal that you're creating. Yeah, and doing that from the ground up, right? It's like every layer in the system try and get rid of noise because what we oftentimes see, again, is an approach in the industry. It's very difficult if you use kind of a generalized approach of creating a pile of anomalies and then trying to harvest out of that five signals that are valuable.
The system is noisy at its core at its base. Well, generally, yeah, you can pick five things that might be interesting, but they're not necessarily the five things that could kill you. And so for us, we fundamentally believe that at every layer in the system, you do noise abatement so that as you work your way out, you have a tractable problem in actually identifying the five things that matter the most. But from an attacker perspective, putting on your glasses and doing the work when the attacker is lenses, that you've picked the things that are most meaningful for the customer.
Right, that makes sense. We alluded to generative AI earlier here in our chat, but I'd like to get your opinion of where you think, if at all, that technology, let's call it a technology, I guess, that technology may assist with the sort of overall effort from a cybersecurity defense point of view. I mean, go ahead. Yeah, I think generative AI, in general, if you think about it, what is generative AI good at?
It's basically, if you take a large language model and you train it up, it's very good at being able to discern two very different ways of asking the same questions, or effectively the same question, right? Because humans can be creative in their use of language. And so the notion of having it be an interface to the analyst, as the analyst tries to prosecute an incident response session and investigates the session, makes a lot of sense. And certainly a lot of what you see out there in the industry from Microsoft's security copilot to Google's SecPong to CrowdStrike's Charlotte AI, these are all manifestations of the same thing, which is how do we take the data that we have, put them in touch with a large language model and allow the user to not have to be as knowledgeable about our data, as knowledgeable about exactly what they should, what opportunities they should pursue next and how the model gives suggestions on how to do that.
I think where these models do ultimately have a problem, as we kind of know from the general news out there, is that they hallucinate. We like to think of large language models as capturing truth, but if you think about it, if you captured the entirety of the corpus of data on the internet and you asked a particular question, there may be 80% of the experts might answer it one way, 20% of them might answer it a different way. Large language models are constructed to give you that variability, to not totally hide the 20% tail. And so sometimes they'll give you the 80% answer, sometimes they'll give you the 20% answer.
Right. For instance, you don't really have consistency from one time to the next, even when you ask the same question, because there's a degree of entropy built into the system. Right. Or control in any way.
Control, yeah. And so if you think about wanting to use LLMs for automation purposes, that's actually not that great, because when you automate something, you want consistently the same behavior and you want that to be efficiently done. And LLMs are inconsistent and inefficient, but they are very good at, is that they are very malleable and adaptive. So as an augmentation to a human being, doing incident and as a check, as an AI assistant, effectively, the intelligent assistant, I think they make a lot of sense, as a means of potentially producing automation, producing the initial cut at, hey, here's a playbook that I think we should run with a human.
So looking at that playbook and going, yeah, that looks right. And then subsequently, when that playbook is run, the LLM is no longer involved. The LLM was simply involved in the creation of the playbook, with the human observing that the playbook is correctly coded. And then henceforth, the playbook is simply run without the LLM in the loop.
I think that makes sense. But I think in automation cases, when you look at source and other kinds of products like that, similar to kind of what you see is in the GitHub thing like that, that it basically allows you to kind of create code. Playbooks are just a form of code. So I think it can be used as a means of allowing people to automate more of their world, but again, not to run the automation, but to help create the automation as a first stanza.
One of our huge vulnerabilities, as you know, is in transitive dependencies within, you know, the code libraries. Is there a, you thought about a use case for generative AI in that, in that discovery process? I know we have scanners. Yeah.
It certainly can. But I think there are other techniques that are probably more amenable to it. Again, can you solve certain problems with generative AI? Yes.
Are there better techniques for solving those problems? For a lot of them, the answer is also yes. So I don't think generative AI contributes that positively to managing those dependency chains and divining truth out of them, because I think that's a simple graphing structure, ultimately, and doing graph analysis is kind of a well-known and well-understood problem that doesn't need to involve the jankiness that comes with, and the inconsistency comes with them. In fact, I think there have been certain examples out there where LLMs have exacerbated the problem.
And let me describe to you a very simple scenario. LLM, you can ask an LLM to create sample code for you. It'll create the sample code. And inside that sample code, they'll say, yeah, include these packages.
Those packages don't exist. You know, this goes back to the hallucination problem. And what people have noticed is, oh, well, people will simply go implement the code that the LLM suggested to them, which has now referenced the package that doesn't exist. And so if a bad guy can simply provide a package in GitHub, that'll just get automatically pulled in.
And so now you've effectively inserted yourself into the software build chain in a way that clearly the creator of the software package did not intend. And so I don't think it really solves problems there. And I think, again, think about what it's good at. It's malleable.
It's adaptable. You can integrate a lot of different data sources that may have, you know, API dependencies and stuff like that. You can use that to interact with a human being who speaks just normal, non-geeky English or other languages. Those are things that's good at, right?
And exposing the realm of possibilities. It's good at. And that's a fair bit. Like trying to get it to solve world number by solving every security problem, I think, is the hype cycle thing, right?
It's where, like, it can do everything. And then we're going to get it to, eventually we'll get it to the trough of disillusionment, as Gartner calls it, and then climb our way back out of that in terms of actual use cases that it can do. Right. And I'm conscious of the time here.
So I wanted to get your take on, if you were to advise a customer, you're obviously well conscious and aware of the breadth volume that you seem to be under here and have been under for months. But it seems like that trajectory continues to bend upward. What's the most important thing you would advise somebody who is trying to figure out how to improve their cyber security posture? I think they're, they're handful of things.
I won't count them out until after the fact. But I would start by saying, you know, adopt new systems, particularly commission critical things. You need to kind of have your eyes open when you're adopting new systems. When you adopt, when you decide to just, you know, put some stuff out on AWS or an Azure or, or put all your key data into a SaaS application without really understanding the, you know, how to control that data, how to secure that data, how to monitor it for attacks.
I think that that is something that we see a lot of companies voice that on their security teams. We've just made decisions to the agile. Let me put stuff over here and here and here and here. And now the security team has to kind of deal with it.
And so what we try and do for customers is just reinforce to them that each of these things has its own attack surface. And you need to be aware of it and you need to be able to manage it and you need to know what kind of your system critical things are. Above and beyond that, as you pointed out, there's both the volume of alerts that you need to deal with. So start, begin with the end in mind, begin with the realization that you can probably only investigate and many things a week.
How you construct a system that just brings that many things to the fore and has a very high likelihood of including in that actual attacks that are occurring. And I think the other thing is it's not just a volume of alert, but the breadth of the alerts. So if I give you an alert from, you know, some, I don't know, key store in AWS versus a, you know, a big query system problem in GCP versus some, you know, Azure AD, SAML identification thing versus an on-prem volume scanner firing something, right? Just a breadth of what a human being needs to understand and how these systems operate can itself be overwhelming.
However, if you don't stitch these worlds together, if you treat them in isolation from each other, there are plenty of examples, both amongst nation-states and kind of ransom ops people that are kind of carrying out these modern ransomware campaigns, that take advantage of the fact that you are viewing these systems in isolation, that attack something in one part of your estate, steal something from there, leverage it in the next part. And so if you look at each of these parts in isolation, there's not enough signal there for you to actually be able to define that attack is occurring. It's incredibly important that you stitch these worlds together, because the attacker is already doing it for you. If the defenders just use these silos to be protected in isolation from each other, I think that ends up being a real problem.
And as I see the trends moving forward, I think it will become a bigger and bigger problem. I'm sure it seems what you just said is certainly accurate from my combo point of view over here, too. I mean, it's just interesting to me that we seem to, the industry as a whole doesn't seem to be, I don't know, same sense of urgency that if you observe this from a mile away and think, boy, this looks like a dumpster fire, how can people run around? Do you feel kind of that same way or no?
Yeah, I think that's the case. But I mean, the way to look at this is you think about through the history of mankind, you think of all the grift and all the con man, that would sell you a magic elixir or that would have some kind of grift where they would get some money out of you. You take all of that creative energy that mankind has spent trying to cheat each other out of things and trying to get one over on each other. And in the world that we live in, now all of it has moved online.
Right? And so the entirety of criminal acts have now pretty much kind of moved online. Why? Because A, it's globalized.
You can be sitting in an internet cavity in Kazakhstan and attacking some company in Houston. And D, we're in the days of old when you try to grift and it didn't work out. You might at least end up in jail or something might break your fingers. Now you live in a world where there is, you know, there are no repercussions.
You can try this from a far away place and it will either succeed or fail. And if it fails, you'll be off nothing in terms of your personal thing. So I think we just need to be kind of realistic. You know, as long as everything worth dealing is available online and as long as we have both nation-states, you know, in various forms of cold and hot wars fighting each other.
And we have systems that are priced with agility and they're willing to kind of do things quickly and change things quickly and morph quickly rather than systems that remain stable and can be hardened for long period of time. This is the world that we live in and that's just a practical reality. I don't think in my lifetime, there will certainly not be any problem for people who are cybersecurity specialists and have a job because it'll continue to be a growth industry. I'm afraid you're right.
So with that, Oliver, I think we'll close out today's podcast. It's been an absolute pleasure chatting with you. You've got amazing sort of insight and reflex here in the space, obviously. And I would love to, you know, sit down again in four to six months or so and kind of see what the world has brought us during that interim.
Sure. If you're open for that, that would be terrific. Sure. I have been accused by both friends and foes of having the opinion and everything.
So yeah, I'm always open for a conversation. All right. Great. All right.
Thank you again, Oliver, to have a colleague, chief technology officer for Vector AI and we hope you enjoyed the 30 minutes or so that we spent today and look forward to seeing you again next time. Until then, I'm your host, Steve King, signing off. Thank you for joining us for another episode of Cybersecurity Insights. You can connect with us on LinkedIn or Facebook or send us an email at social at cybered.io for more information about the podcast.
Visit cybered.io forward slash podcast. Until next week, stay safe and secure and we'll see you on the next episode of Cybersecurity Insights.