Modern software relies heavily on open-source dependencies, often pulling in thousands of packages maintained by developers all over the world. This accelerates innovation, but also creates serious supply chain risks as attackers increasingly compromise popular libraries to spread malware at scale. Firas Abu Khadija is the founder and CEO of Socket, which is a security platform designed to protect software projects from open-source supply chain attacks. In this episode, he joins Josh Goldberg to talk about his career in open-source, open-source supply chain attacks, practical security lessons, the expanding attack surface in software development, and more. This episode is hosted by Josh Goldberg, an independent full-time open-source developer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ESLint, a powerful static analysis tool set for JavaScript and TypeScript. He is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a co-founder of SquiggleConf, a conference for excellent web developer tooling. Find Josh on Blue Sky, Fostadon, and .com as Joshua K. Goldberg.
For us, Abuka DJ, welcome to Software Engineering Daily. Thanks, Josh. Glad to be here. We're excited to have you. You have been in and around open source and general security practices for quite a while. Before we dive into you and Socket, can you tell us how did you get into coding? Yeah, I got into coding when I was in high school. I wanted to build a website to collect my favorite Flash animations. So I was kind of born in the era of Newgrounds and E-Bombs World and Albino Black Sheep and just all these kind of, I don't know if folks remember these or if they're too young, I don't know the audience of this show. But yeah, I thought those things were fun and I wanted to kind of collect them all and put them onto one website. So I did a lot of downloading of...
those SWF files from other people's sites and then re-hosting them on my own page. And I had to learn PHP to do that and MySQL. And so that was kind of my first foray. And then you went into Stanford for computer science after that. I mean, yes, I did go to Stanford to study CS. My high school didn't have a CS class. So I kind of was just self-taught with PHP up until that point. But learning CS at Stanford was amazing. A lot of the other majors at Stanford, they don't necessarily emphasize teaching well, but that's one thing that the computer science department really stands out in. They have just like a ton of support, like other undergraduates actually are your TAs and like help teach you. And so I learned a ton. I remember my first.
class I took there, it was using C++. And I remember my first reaction was, how does the computer know that these words are variables if they don't have dollar signs in front of them? Because in PHP, every variable has a dollar sign. And so my mind was blown. I almost spent too much time in PHP in high school. And it took a little bit of unlearning to realize, oh, wow, CS and programming is this really broad thing. It's not just PHP and MySQL. Yeah. Do you ever go back and look at the PHP you wrote in high school? I have actually done that before. And it's hilarious. I didn't even know about functions. Even after years of writing it, I literally just pasted everything multiple times if I needed to do it in more than one place. So it's...
It's horrible, but it works. It kind of didn't matter in some sense. Like I built a site that got, I think at its peak, it had like 600,000 annual visitors watching all those flash videos. And I don't know, it's kind of funny how, I think the lesson from that is that you should just jump in and do stuff. And I mean, I obviously would never write code like that now, but like if you let doing things the right way get in your way, it can kind of take the fun out of it and like stop you from just kind of, I don't know, catching the bug. And I caught the bug. What really got me excited and kept me going. And it was this idea that I can put code online and while I'm sleeping.
It's like working for me. It's like serving visitors. People are coming to the site, like having a great time. And I'm literally asleep. It's like that scene in Fantasia with the brooms and the buckets. And like, I don't know, it's like automation. I don't know. It's just sort of this cool idea of like this thing is just out there. And I think at one point I put Google AdSense on the site. And so like, I was like, oh, I'm sleeping. And it's like making me money. You know, it's such a cool idea. Yeah, I don't know. That got me really into stuff in high school and college. Believe it or not, it's not the having excellent clean code that gets a lot of newcomers in. It's the results in being able to have that superpower. Yeah.
Yeah. That said, I also think it's good to use functions. I wouldn't recommend anyone copy what I did in those early days. Good to know. So that was your first popular website. What was your second popular or hit project or website that got released? So while I was still in high school, I did another site that was around sharing my notes for my AP classes that I was taking with other students. So you could go there and access.
outlines of different chapters in the book. It was kind of like a SparkNotes for AP courses. And that was great because a lot of high schoolers didn't want to read their textbook. And so that was something that got quite popular. That's also when I started learning about SEO and just like all the web search type stuff. And like, how does Google work? How do you like properly semantically build web pages that are going to be indexed by Google? Google is going to love indexing and sending visitors to you. So that was a cool way to learn. And then like after that, when I went to college, there was one other site that I did that was maybe notable, which was called YouTube Instant. And this was kind of the result of a bet that I did with a college roommate of mine. So Google had just announced this thing called Google Instant, which was a way to kind of
it's kind of like autocomplete on steroids so as you type like letters into the search box instead of just showing you like here are five possible searches that you might want to do it would actually take you to the search results page of the first suggestion before you even hit enter so like you type a and then it's like takes you to this actual search page with like the 10 links for apple or whatever it thinks you were searching for and so i saw that and i was like oh that's cool i bet i could do that for youtube and it would be even more cool because now you're typing letters in and you're getting like flashes of like different videos coming in and like it's almost like channel surfing like youtube and so i bet my friend i could build it in an hour which was kind of completely ridiculous but i just thought like oh let me just I don't know why I did that, but I did. And then he was like, yeah, I'll take you up on that. And then I actually did build it in three hours. It was very simple. It was just using like a YouTube API and like putting a video embed into the page. No fancy web frameworks, no futzing with like getting typescripts and reacts and all that stuff set up. It was just literally like one JS file and one HTML file. So very, very simple, pretty horrible code as well, but got that up. And then I put a tweet out and a Facebook post out and went to sleep. And I woke up the next morning and it was completely viral. It had gone like across the internet overnight while I was sleeping. And I don't really, to this day, really know why. I think if I had to guess it.
was because it was pretty fun. Like once you used it, people wanted to share it. But like, it was also because it kind of like piggybacked on Google Instant. And they had spent two years engineering Google to support that level of new load on, you know, just in terms of number of search pages being loaded as literally everyone typing letters. It's like 10x or more of the search volume that they had ever had in the past. And so they like hyped up how much like engineering effort they put into this. And then like, lo and behold, some college sophomore comes along and is like, I built the same thing for YouTube in three hours. And then that somehow like unintentionally like became this like media story. And then it was like fueled even further because the YouTube CEO tweeted to me and said like, hey, this is great work. Do you want a job at YouTube? And then like that caused like a whole nother...
burst of attention people were talking about how like the resume is dead and like building projects and putting them out there is the future of getting hired and like all this thought pieces came out of it and i was just like kind of swept up in this media storm basically so it was kind of a fun experience to go through as a sophomore and like kind of trying to not like say something stupid while getting interviewed by nbc or whatever you know so it was pretty cool that sounds like a very thrilling ride are there particular lessons you took away from it that you still use now as a ceo on the other side of the hiring table huh yeah i mean i think the big thing is like just launch like just ship things don't think too much about it like build stuff in public i mean if i had sat around and like i don't know tried to use the right framework or tried to and it's funny i say this like i'm actually good at this but the reality is I'm more saying this as advice to myself to remember, because I also tend to like over-engineer things and like want to do things the right way. And like sometimes don't ship. And like what I think worked in this case was I just like put it out there. It didn't really matter that it was like pretty bad code. I like put it out there and then I improved after I saw that people actually cared. And so that's the right way to do things. If you're interested in like finding something that works or that resonates with people, it's just, it's really about like the number of things you put out there, not like sitting around in a room by yourself, trying to like think of the perfect idea. And that's a lesson like I've had to learn again over and over in my career, including like literally when I started my first company and we kind of did the opposite of that, where we literally just sat around and like built.
some cool technology for about eight or nine months without really putting it in front of any users. And then at the end learning, actually, nobody wants this. We just spent nine months building. It's a cool science project. I learned a lot. I'm glad I worked on it. But nobody wants to buy this or use this. So I could have probably learned that really quickly if I just talked to people or had put something more minimal out there. You are not the first CEO to mention this on this podcast. Yeah, that's a really important learning. Yeah, there was even a guy on Hacker News when YouTube was on the homepage. It was actually on the homepage a couple of times, multiple times at the same time. And then one of the comments I remember was this guy who, I don't know who he was, but he was really upset that...
my site was on homepage because he said, he was like, I've been building the same site for three months and it has way more features and it supports like filtering and sorting the videos by views and by different things. And he kind of was explaining all the things it could do that mine couldn't. But like, then someone replied to him and was like, you should have just shipped. And I saw that. I was like, you know, evidence of this lesson literally in that thread. And I mean, obviously it felt kind of bad for him. He put it in a ton of time and like, but it sort of proves the point of he should have just shipped and no one really cares about all the advanced options he was adding and he should have just put it out there sooner. Yeah, it's a pity.
You've shipped quite a few things. We could take this either through your career or through your open source projects in general. Let's talk about the open source projects first, because they're quite visible. Is this a continuation of the same mindset of you just shipped a bunch of NPM packages? How does that play out? Yeah, I mean, so I got into open source originally because after that company I mentioned, it was called PureCDN that I worked on for about eight or nine months. It was, we basically, we're going to shut it down. And then we got really lucky and kind of out of nowhere, Yahoo offered to buy the company. And really they wanted us to just join the team and like work on JavaScript and video players and stuff like that at Yahoo. So we took the offer because it was me and my two friends and we were like, well, yeah, this sounds like a good way to.
wrap this up and make something of this unfortunate situation that no one actually wants what we built. Maybe we can go to Yahoo and they'll use it. But they ended up not using the code really at all. They had bigger problems than reducing bandwidth costs of video. And so we ended up just kind of helping with more fundamental things like making their video player load quickly and work on mobile and stuff like that. But I did feel kind of sad after all that work.
science project I was talking about, like all this cool technology and stuff we had put into it, like didn't get to see the light of day and would never get to see the light of day because now they owned it and it was all proprietary code. And so after I left, I worked there for about a year and I'm proud of what I did there, but it was also one of those things where, you know, it's a really big company and it was like hard to ship things. I felt like I was fighting the company to like ship things, which is crazy. At one point, my manager said to me, you need to do less. Just like, stop trying to do all this stuff. Like you're making my life hard. Thinking like, this is not why we required literally.
The CEO of Yahoo, Marissa Meyer at the time, told us, and she was on a startup acquisition spree during this time. She told us, the reason we're buying you is because we want you to inject startup energy and just this attitude of getting things done and shipping stuff into the company. And this is why we're bringing you on. But we were just too few and we were on a much bigger team with the old school folks that were there before. And so we did some good stuff and there were some really smart people on the team. But ultimately, I think the organization couldn't really handle it. The old blood one rather than the new blood, if that makes sense. So I left and then I wanted to play with some of the same ideas of what the original company was doing, which was about building a peer-to-peer CDN. So think like...
a content delivery network like Cloudflare or Akamai, but powered by end user devices. Like your laptop is in a network, kind of like BitTorrent. And if you're watching a video, you're going to serve that video to other folks that are coming to the same site. And it's sort of like a big peer-to-peer network. And that could speed up sites in some cases. And it could also massively reduce the cost to host sites, which before Cloudflare was really popular, CDNs were actually quite expensive before they started giving everybody bandwidth for free. And so this was where the idea came from. And so after I left Yahoo, I just decided, oh, it'd be cool to like build that again, but in an open source way so that no one can ever like...
take it away from me or from the community. I want this to be open. I want anyone to be able to use this. And so that was like why I got into open source. I mean, I had always thought of open source maintainers as this sort of crazy, awesome level of like programmer that like I one day wanted to aspire to be. You know, when I was learning the code, I was almost afraid to look at open source packages because it's like all this crazy advanced code that how could you understand what it's doing? So I kind of always thought it'd be cool to do that and like get back in that way. And then I had this opportunity to do it with WebTorrent. And so that's kind of what came out of all this was I ended up building a torrent app that could work in your web browser.
and do peer-to-peer, like from my browser tab to your browser tab without any software installed on the computer. And so you could really get like BitTorrent experience like on a website. Which is amusing now because BitTorrent has BitTorrent web. Yeah, and actually they use WebTorrent under the hood. The protocol that we developed, like all the major Torrent clients actually support WebTorrent now, finally after like 10 years of trying to get adoption of this standard. So it's pretty cool to see finally that's actually like the vision has actually kind of happened where any Torrent client running on a desktop can talk to a web browser user across that bridge, which is pretty crazy. I mean, when you think about it, that that's actually possible with WebRTC. As an open source maintainer.
How do you feel when one of the projects you lead or maintain or worked on becomes successful in this way? That's a fun one. So at first it's like awesome, right? I mean, you get to talk about it. People are interested in it. You get invited to speak at NodeCon for JSCon for whatever. And I remember at that time in my life, I was my early 20s and it was super fun getting to go travel the world. Like I went to so many countries I've never been to before, met people. And it was, it was funny too, because people, they would use me and a few other early Node folks as the mad science track of the conference. So they'd have all these talks on like React and kind of practical things like, you know, how do you deploy microservice and this and that and the other thing. And then they wanted to have like certain talks that were like, you're never going to use this at work.
but it's just cool. And that's what they would invite me to be the entertainment, I guess, in that sense. And so at least I think that's why I was invited to those. And so it was just fun to be able to show people what I was building. And it was also very in public. Because honestly, for the first six months of talking about WebTorrent, it didn't work. It was literally in development. And I was talking about it and explaining how WebRTC worked and how this was all going to work and doing some cool demos of WebRTC for people during the talks, some live demos and interactive things with people all interacting on a whiteboard from their phones and on my screen and over a peer-to-peer network. And it was really cool, but it wasn't WebTorrent. And so I was definitely doing it in the open and it was really fun.
But then when you get to a certain point in your project, like what ends up happening, I've seen this happen so many times, not just with me, but with like every, basically almost every maintainer, but not all, but almost every maintainer that I know is that they eventually just get burned out of doing it. Because at least for me, what happened was I felt this incredible responsibility to like fix every issue that was reported on GitHub. And at first the volume was really low. It was like someone would open an issue and you'd be like, oh my God, this is so cool. I have a user using this. And you're like so happy. And then eventually you get to a place where you wake up every morning and there's like 40 issues open. And you're like, oh my God, this is not sustainable. And then they were all issues too. Like they weren't even, it was like, I kind of forgot why I got into it almost by the end. Cause I was like.
Someone's reporting about, oh, it doesn't work on Arch Linux version, whatever. And I'm like, okay, I don't even care. I should have just said to them, like, you submit a PR if you care about this and I'll review it and merge it. But instead I took it like, oh, it's my job to go and fix it. And so I very much burned myself out by the end and like basically needed to walk away and be like, add other folks as maintainers and just kind of like go do something else for a while. There's a great blog post that you've, I think, alluded to in the past from Nolan Lawson, what it feels like to be an open source maintainer, talking about how it's like you have this giant stretch of people in front of you and there's absolutely no way you can address all of them. And it feels awful on both ends. Yeah, it definitely feels like that. You want to do the right thing and help them and you feel this obligation.
Cause it's like your code and they found a bug and they're right. There is a bug and like, you're correct. And so therefore I must fix it. But like, actually there's not a contract saying that you have to do that. It's like, I put this code online as a gift to the world. I didn't promise it would never have a defect. And so it's actually okay. It's actually enough to give a one-time gift. You don't have to give a permanent SAS subscription of your time as a gift to the whole world. So I think that was the realization that I like finally got to, but after getting super burned out. Let's talk about that dark side of open source because it transitions really well into what you've been doing recently. Specifically, let's talk about security. What is socket.dev and what's the open source problem that it's trying to address?
So open source when it works is amazing, right? You can just go out and grab any code from the internet that's written by somebody you don't know. You don't know who they are. You probably didn't even read the code and you can just pull it in and run it on your computer and saves you a ton of time, right? It's this amazing like buffet of unlimited, you know, all you can eat code that you can just pull from. But there are bad guys, you know, there are criminals that have realized that actually getting access to an open source project and putting malicious code into the package is a great way to attack a bunch of people at once at a really large scale. And so we've seen a lot of attacks of this nature over the last several years.
for all kinds of motives. Sometimes it's politically motivated. It's countries attacking other countries or trying to attack US companies or things like that. Other times it's just basic cryptocurrency criminals that want to just steal people's crypto. There's even been cases where people have, like, they just, they were a good maintainer at one point in time and then they just, something just snaps in them and they want to just, they want to watch the world burn. And so they'll put malicious code into their own package that has been trusted for super long. And so it made me start wondering, like, why is there no way to stop this? Why is there no way to catch this? Especially after like the fifth or sixth time I saw this happen. You know, I mean, the first time I saw this happen is in 2017 with EventStream getting compromised. This was...
a package where a new maintainer was added who basically said, hey, can I help maintain this package? I noticed it hasn't got any updates in a while and they got access. And then after about 30 days of doing some good changes, they put in a malicious change that stole a bunch of cryptocurrency out of a specific application that was targeted. So they obfuscated the code and then it only triggered in one specific Electron app. So everyone else who had the code just sort of did a no-op, but in this one app, it would trigger and steal all the user's money. So this kept happening. And then I looked around and I was like, why is no one doing anything about this? Is this just like the way the world works? I was also confused. Why is nobody reading the code that they're using? And then I realized I'm really weird. And like, I read every one of my dependencies before I use it.
I mean, like maybe not anymore, but like when I was working on WebTorrent, certainly it was a small enough surface area that I could actually do that. And then I realized like no one else really does this. Like it's pretty rare thing. Like some people do, but it's pretty rare. And so that's why like there's really not much of an immune system against this. Usually what happens is like the attacker makes a mistake that ends up showing in a log file or something. And then the community just accidentally discovers it. And that's what kept happening. It was always like, oh, we accidentally noticed that like this package has been stealing everyone's information for the last two weeks. And that's when I was like really worried because I'm like, how many more of these do we not know about if like we're just accidentally finding these? So in the event stream example, let's give you an example of how we found it as a community.
It turns out the attacker, they obfuscated their code and they were using the crypto module in Node.js to decrypt the attack code. But they didn't know that in literally a day or two after they backdoor the package, that Node decided to deprecate the function that they used. And so everybody who was using the package started getting this deprecation warning. And then they traced it back to this chunk of code that was added into this library. And so that's such a random accident. If that hadn't happened, it might've gone for weeks or months without being discovered. And this pattern just repeats in every one of these attacks. And so that's kind of like when I was like, dang, this is a problem. And no one seems to know how to solve it.
So what are the techniques that we can use as application developers, separate from, say, Socket, just to kind of harden ourselves against things before we go into specific services? Yeah, yeah, for sure. I mean, the number one most important thing that everyone should be doing, and everyone is probably already doing, but if not, you should use a lock file. So your package manager, probably, depending on the language, but almost all of them support a lock file of some kind. And this is very important because it locks down the specific versions of every package in your dependency tree so that when a new person on your team clones the code down and runs the install command to install those packages, they're going to get the exact same set of dependencies that you had when you built the application.
And so there's no randomness or there's no new versions being brought in unintentionally. It's very explicit. There's also a record of what versions of every package were in use in the Git history. So everyone knows what was in use at every point in time in the application. That's the number one thing you can do. That obviously doesn't stop you from installing a bad package, but at least you're not just pulling in whatever happens to be the latest version at the time you run the install command. And I want to emphasize too, it's important that you're using a lock file and not just pinning your direct dependencies to a specific version. If you pin your direct dependencies, that's a good start, but...
that doesn't lock down the transitives because those can also have loose ranges. And then the package manager will go ahead and install your direct dependency at that exact version. But then in the transitive dependencies, those can be loose ranges that it'll just pull the latest stuff as well. So you're still subject to really, you're going to get whatever code happens to be there at the time you run install. So that's like, I think that's the biggest thing. What's another good tip for us? I mean, really the best tip is to use Socket, but I'm not just saying that because I started the company. So I mean, forgetting about the specific solution of Socket, like really you want to have some process for vetting dependencies. You want to know what you're depending on. And I think what really helps is having a bit of a mindset shift around.
how you think about dependencies so today a lot of people think about them as you know just this magical code that just comes from the cloud or from the sky somewhere and then it just sort of solves your problems for you and they assume because it's open source that it must be safe or that someone is vetting it right but that is really not the case like i mean everyone is assuming that someone else is vetting this code right like everyone has that assumption oh yeah it's open source like someone will vet it everyone is making this assumption and then you actually realize very few people are actually opening up the code and looking at it like shockingly few right this is really obvious when you look at the kinds of attacks that have been pulled off by these attackers and not discovered for you know like i said weeks sometimes months they're so obvious these attacks they're like if you just open one of the js files or one of you know one of the source code files you would see like just it doesn't look like normal code it looks super obfuscated or there's these giant base 64 strings in there there's you know crypto module being imported to decrypt the code or there's eval happening you know like a string is being executed as code or there's obvious things like the environment variables are being collected and then sent off in a network request to a suspicious URL. It's like horribly obvious when you look at the file and then you realize, okay, no one is looking at this stuff. So this is how it's able to sit in plain sight. It's interesting because Linus Torvalds, you know, the creator of Linux has this law, I think it's called Linus's law that is quite, quite, quite well known. I think it's the idea that with many eyeballs, all bugs are shallow. Hope I'm getting the quote right, but something like that. And the idea is that with open source software, right, eventually with enough eyeballs looking at the code.
all bugs will be discovered at some point and then be fixed. And it's true. And he's really highlighting the value of open source code versus proprietary code and how when you have open code, people can find these things, they can fix these things. And that is all true. But the question is on what time scale, right? How long is it gonna take for the community to find the bug, right? And in the case of a malicious actor putting malware into a library that everybody depends on, the time really matters because some of these things have 10 million downloads a week. And so if something's compromised for even 24 hours before people notice, you're gonna have a lot of affected people, a lot of affected applications and companies. And so I think-
that's really the caveat is like how long until detection. And before we started Socket, what we saw was there's a great research paper published in the Usenix security conference a few years ago that found it was around like 200 plus days for a malicious package to be discovered by the community and taken down. So really, really bad results, you know what I mean, in terms of hoping for the community to kind of self-correct in this way. So I think the main thing is just shifting and thinking of open source code, not as like someone else's problem, but actually treating it as your problem. Like it's part of your application.
At the end of the day, all this code gets bundled together and run in a single process or in a single binary or in a single application. And so it doesn't really matter if you wrote this code and then someone else wrote that other code. Ultimately, you're shipping all that code together to production and you're responsible for what it does. It has access to all your user data. It has access to all the crown jewels. So at the end of the day, it's your code, even though you got it from GitHub or you got it from PyPI. So we've had this problem where we have an absurdly fastly growing amount of other people's code in our applications. These days, we have so many features as developers. We expect hot model reloading and deduplication of packages and all this.
that there's just an inhuman amount of code to look through. So having services look at that code for us seems like a natural next step. And you mentioned some kind of somewhat straightforward heuristics, like importing the node crypto module or sending your process environment variables in a network request. Stuff that's almost humorously evil. So how does Socket take a look at the other people slash our code and determine what is something to be complained about? Yeah, great question. So when we started Socket, we started with our own observations about these types of attacks we were seeing. So things like accessing environment variables, accessing the file system, accessing the network, especially when that was a new behavior in a package. Like, you know, it hadn't done that for...
any other version in its entire history. And then suddenly now there's a new version that came out yesterday that needs all these capabilities, all these permissions, you know, that weren't needed before. And so that's kind of what we started focusing on was detecting just that introduction of new risky capabilities. Oh, install scripts is another good one. Not all package managers have this concept, but NPM is a famous one that does. And this is a way to automatically run code on the developer system at the time of package installation. And while it has legitimate uses, almost every single piece of malware that we were seeing on NPM took advantage of this functionality.
The presence of one of these didn't necessarily mean you were dealing with malware, but all malware pretty much had that characteristic. So you could start to build up these heuristics of like what looks like a suspicious change. And then, you know, you could sort of alert on them. So if you're about to do an NPM update or you're about to, you know, pull in a new package for the first time, you could, you know, see what socket had identified in that package. But then what we found was a lot of teams wanted something even more simple than that. They wanted us to just have like a yes, no, like a Boolean. Is this package malicious or not? And so right as we started noticing that desire from folks, we got our hands on GPT 3.5. We were playing around with this idea of like, can...
actually look at code patterns and figure out what the code is doing. So not like the main way people were using AI at that time was just sort of to build these little chatbots. Like you can chat with your database or whatever, right? These types of silly things, in my opinion. We were really asking, can they understand the concept of maliciousness? And can they look at the code and tell you whether there's something maybe worth a second look? And so it didn't really work that well with GPT 3.5, but then GPT 4 came out and we got access to the API while it was still in private beta. And then we found...
holy crap, this actually works really well. So what we had to do though was because of cost, right? It's super expensive to use LLMs, especially if you're trying to scan every open source package. NPM alone has like 2 million packages, not to mention all the other ecosystems. So we had to decide when it was worth the cost and the expense to actually do that type of analysis. And so we used some of those static signals that we had developed before, like does it have network? Does it have file system? Does it eval a string? Does it look like it has obfuscated code? Then we would take just those parts and kind of feed it into the LLM. And that's how we...
you know, we built the system that we have today. It still has false positives, but if you put a human in front of it to kind of do the final sign off, you can actually get like a really high signal from it. And so that's kind of what we, one of the main things that Socket provides for our customers today is like you can get this like really high signal feed, whether packages are malicious or not, and then new ones that are being backdoor that no one knows about yet. So like if you open up our feed right now, you'll literally find something from 30 minutes ago that was just published that's malicious. It's still live on the package manager registry that you could install if you were so unlucky to do so. And yeah, and we report them when we find them, by the way. So we try to get them taken down to protect everybody, but we often get really, really slow response times from the different registries. Some of them are better than others.
PyPI is pretty good these days, but like a lot of the other ones are very slow. And so you'll have things where like there's malware from nine months ago that's still live that they just don't take down. So you have to use something like Socket or some other database of this knowledge to kind of protect yourself. So let's say that I'm a company. I'm an enterprise. I have software with dependencies on, let's say a couple of registries, NPM and PyPI. What would I do? How do I use Socket to save myself from these ridiculous malware attacks? Yeah. So easiest way is if you're on GitHub or one of the other major source control systems, GitLab, Bitbucket, you can install Socket from the marketplace. And then we just go right into your GitHub installation and we can start.
observing all pull requests and all commits that are happening on all the repos and we scan every single commit that we see so we can do two things one is you first first of all you just get visibility like what dependencies am i using across all my repositories and you can have like kind of a snapshot at every point in time as that changes so if you ever want to know in the future like hey did we ever use this bad package at this bad version number you can like look it up and see and then you can also get a inventory of like you know all of your existing risk so any malicious packages or vulnerable packages or you know other risks that you have within your current set of dependencies you'll be able to see that but the most important part is the preventative piece so that's where you know, in a pull request workflow, whenever a developer is adding a dependency or updating a dependency to a new version, those are the two kind of moments where you're bringing a new third party code for the first time. And so Socket will look at all those changes. And if there's any risks, it will leave a comment in the PR and just tell, you know, all the folks on the team, even the code reviewers as well, as well as the original PR author about the risk that was identified. So, and it's all configurable. So obviously malware is frequently what folks want to block, but you can even use it for things like, you know, I don't want to use a package that hasn't had an update in five years. You know, that would be a bad idea. So, you know, or I don't want to use a package that has obfuscated code in there unless I...
like click the link that Socket provides and go and look at why is it doing this? You know, it doesn't make any sense. So you have folks set it up in different ways and you can kind of customize it to what you care about for your team. Sure. I imagine, especially in a time where so many companies and open source projects are using tools like Renovate and Dependabot to automatically send PRs for new packages. Something that's blown my mind is most of the time by default, these PRs come in as soon as possible. A lot of default configurations don't say wait for seven days or upon package release.
This feels like something that Socket would be very useful for to make sure that you're not installing a malicious package immediately upon its introduction. Yeah, that's a great point. One of the things that we've actually seen with a customer is dependentbot opened a PR, and then Socket came in and left a comment and said, do not upgrade this to this package, it's malicious. So it was like a battle of the bots. One bot is like, upgrade, and the other bot is like, don't upgrade. So yeah, no, totally. It's a really good point. And some folks do try to do a delay to sort of protect themselves from these things. So that is another piece of advice that I would recommend. If you have an easy way to not take in updates until 7 or 14 or 30 days have gone by, that's a really great way to avoid.
some of the subset of these attacks. Like it won't avoid all of them. As I mentioned, some of them stick around for hundreds of days, even after Sockets identified them and reported them. So it's not foolproof. And it also means that you're also not going to get vulnerability fixes if you're waiting 30 days unconditionally. So you might want to have a more nuanced policy where you wait 30 days unless there's a critical CVE, in which case you might want to do it sooner. So yeah, there's really a lot of options here.
One of the great and terrible points of working in security is that as soon as you fix or largely prevent one style of attack, the attackers will then figure out another somewhat more complex area or style of attacks. What do you think is the next round or the next area that you're going to have to focus on? Yeah, I think that there's a couple ways I could take this. I think I'll maybe mention just the way that we're seeing some of the same types of attacks that have affected. open source package ecosystems starting to come to like maybe less obvious ecosystems like Chrome extensions, Firefox extensions, VS code extensions. So these are also just JavaScript packages at the end of the day, if you think about it. And so some of the same types of supply chain attacks that we've seen affect NPM are now starting to affect all these other ones. So, but they also have in some ways, even their own challenges. If you take Chrome extensions, for example, they're similar to NPM packages in that they run in a very privileged environment. They run in your browser and some of them have access to all site content on all pages that you visit. So it's kind of very similar to a package running on your local machine with all of your user permissions and being able to access all your local files and things like that.
In some ways, they're even worse because there's a certain thing we've seen in Chrome extensions that we've seen a lot less of in open source ecosystems, and that is selling of extensions. So folks will just sell their extension to somebody else. You know, it can have a million installs and they'll just sell it for like 30K or 40K to some random person who offers them money. And I kind of get where they're coming from. A lot of times folks build these extensions and they don't have any way to monetize them. And they're just sitting there and then, you know, they can get a meaningful chunk of money like that. You know, they're just like, yeah, why not? And they often don't necessarily know that the person buying it is going to change the behavior in a malicious way. They often tell the story about how they're going to improve it. They're going to keep working on it and all this other stuff.
We've seen that happen a few times in the Chrome ecosystem to really pull results for everybody involved. So I think that's an area for us that's like, you know, I guess it's sort of expected that attackers are going to go there. They're just always looking for where's the next, you know, way that we can get in into environments. And I guess I should just kind of cap that off. I should mention that we actually just announced this week, yesterday, so July 29th, for those who are listening to this at a future date, that we now scan Chrome extensions. And we're going to be doing VS Code and the rest. So very exciting. New product for us. And yeah, hopefully we can help keep folks safe, you know, even in their browser.
Yeah, I think the other thing that's interesting is AI, just the way folks are using it. So there's a lot of folks vibe coding these days. And one other area that we've seen attackers go is, everyone knows LLMs can hallucinate. And one of the most dangerous ways they can hallucinate is when you're writing code, especially if they're executing that code without user interaction, like in an agent. And so everyone is pretty aware, I think, of the idea that they can write buggy code. They can write poor quality code, spaghetti code. But far fewer people have realized that LLMs can also just write insecure code. Or specifically, they can install dependencies that are insecure. In some cases, they can hallucinate.
package names that don't even exist. They're not even real packages. And so what attackers have actually started doing is like running the LMs and generating a bunch of like asking the same question a bunch of times, getting a bunch of lists of what packages the LM attempts to install. And then they go and they squat on those names. So if you can basically predict what the LM is going to hallucinate, you can go register all those names. And just like you would squat on a domain name, you can squat on the hallucinated names. And then you can basically get remote code execution on anyone's machine when the LM selects those packages. So that's one where it's nasty. And it's very, very much like a 2025 kind of a problem. It's kind of a beautiful attack when you think about it. It's just so clever.
Yeah, I agree. Attackers are always, I mean, that's what they do. Whatever's new, they're already thinking of how to break up. In this AI world where everyone's just trying to go super fast and like, you know, security is an afterthought as it always is. Whenever there's like a new paradigm shift, there's going to be stuff like this and the attackers are going to find it. Like it's always, and that's definitely what's happening with AI. Like security is such an afterthought. It's just like, you know, I mean, another one is MCP servers, right? Everyone is just hooking up. Like they're taking all these API tokens and putting them into like local files on their disk that any malware can just grab, right? Any NPM package that's backdoor can just take those things. They're putting all these tokens into these files and then they're connecting all these, you know.
disparate data sources, like their Google drives, their, you know, their emails, all their most sensitive information so they can talk to it, you know, and have the MCP server, you know, give new tools to the agents. But then, first of all, the files are just sitting there with all the tokens. But then on top of that, you fundamentally have a black box AI model that nobody really knows what's going to do. That's like, you know, has privileged tokens accessing like all this stuff. So it keeps me up at night a little bit. As a member of the defense or the blue team, how do you prevent against these types of attacks?
I don't know if there's a really good way for a lot of this AI stuff because a lot of the adoption is being driven by like company boards and, you know, company leadership. They don't want to be left behind. They're getting asked all the time, like, what is your AI strategy? What are you going to do for AI? And so there's a lot of pressure to like, just tell everybody like to just use all the stuff, you know, and not to worry so much about security. So even formerly very security conscious companies and, you know, CTOs and engineering leaders are kind of like lowering their...
standards a little bit in this rush to, you know, adopt this stuff super fast. So I don't have a great answer for you, like how to generally solve this problem. I think security will get figured out over the next couple of years. Like it always does with these things and it'll be bolted on and it'll be, it'll have a bunch of gotchas as it always does when you do it after the fact. So I don't know. We'll see. We'll see.
I've got one last technical question, just as a curiosity. There are sort of traditional ways to run JavaScript that you've mentioned, Node and NPM. And now there are kind of newer versions of those variants, like PNPM and Dino, that have more restrictive security models. Do you have thoughts or opinions on those in the security landscape, given what we talked about? Yeah, I think that I love more experimentation. I'd love to see the stuff that those teams are doing. I think that PNPM, I believe, now restricts install scripts by default. So that's awesome to see, because like I said, yes, they have legitimate uses. But if you have a backdoor package, if you have a malicious package, the vast majority of them are going to use that feature. And the other thing that's nice about it is it's such a rarely used piece of functionality that...
It's actually not that onerous to just ask the user, hey, is it okay if this package runs this install script? So I think it's really great what they did. And then Dino obviously has like its permission model, which is really interesting. I'm a fan of that. Node has also actually kind of adopted a similar, it's pretty different, but it sort of does the same thing, way of running packages with really locked down permissions on those executions. So it's great to see this kind of stuff. I mean, I will say like, I've really met very few people that actually are able to use that. Like, you know, I don't want to pick on anyone. Node and Dino's permissions model is like, it's not really used.
in my experience, at least in real world applications, it's really helpful when you're pulling a random script off the internet and you want to run it and you don't want to give it access to the network and to your file system and stuff. It's amazing. But like, if you're building a real application, like, you know, that has to do all the things, like talk to the network, read the files, you know, all this stuff, you're going to give it all the permissions anyway. And so I think it's good, but it's not widely used and not really, you know, in its final form yet. More feature-rich experimentation required. Yeah, I think so. And then more work on the developer experience. Like how do you write these policy files? Like the node policy file is really complex format last I looked at it. So I think it will need, there'll need to be some ways to auto-generate that that work really effectively and don't break your application before you're going to see widespread adoption of that kind of stuff.
Looking forward to it. For us in our last minute, I'd like to end interviews with something very much non-technical. Can you tell us about your cats? Sure. Yeah, so I got two cats. One is called Butter and one is called Cream. And they are about one year old now. Actually, they just turned one. And they're amazing. They're super cute. I was hesitant to get cats for a super long time because I'm really busy working on the company and just traveling a lot. And I thought they would be a bit of a burden to find someone to take care of them when we're gone and stuff like that. But my wife was really excited about the idea of getting them. And I've always loved cats. And so she kept talking about it. And then one day I come home and...
in the bathroom, in the bathtub, I find a bunch of litter, cat food, and a bunch of cat toys. And I'm like, okay, she's up to something. And then like the next day when I come home, there's two kittens in the house. So she just kind of like went off on her own and got them. Ordered them on Craigslist actually. So there was someone that was looking for a home for their two kittens and she got sent some videos and then actually had them delivered by an Uber package delivery. So Uber picked them up and drove them two hours to our place. And that was probably a really fun Uber ride for that driver, having two kittens in the car. It's been amazing. They are.
A joy. Cats are fantastic. Since you mentioned conferences earlier, I've found that in cities, for example, Athens, that have a large amount of cats, going out with other conference attendees and speakers to feed street cats is a surprisingly fantastic bonding and networking activity. That's so funny. So they just go out and give food to the cats and hang out with them, the wild cats? Yeah. Walk 30 seconds in any direction from, say, city center Athens, you'll find them. That's awesome. I also found certain parts of Tokyo or certain neighborhoods have tons of wild cats. Feral cats, I guess, going around. I just love cats, yeah. Their behaviors are super cute. They're always up to something. They're always investigating or trying to do something.
something that they're not supposed to. It's kind of fun. Given how much time you spend day to day on people with very malicious intentions, it must be nice to work with a creature where the maximum maliciousness is that they want the tree you're not supposed to give them. Yeah, it's so true. They do get in trouble too. Like if you find ways to climb into things they're not supposed to get to. We just built a catio, which is like a little cat house outdoors over here by my desk. And it's quite big. It's like a eight foot by six foot like outdoor structure. And I just cracked the door and then they're able to kind of go.
directly into the catio, but there's currently a little gap in the part above it. And so they figured out they can climb up the side of it and then get outside. So I'm gonna have to like add another layer of chicken wire on the top to kind of fully seal it with the door or else they're gonna find a way to get up. Crafty creatures. Yeah, for sure. Well, Firas, thank you so much for spending extra time to talk not just about your open source career and journey and what it's like to be a maintainer and all this stuff Socket is doing to help protect open source projects and companies, but also about your adorable two cats. If folks wanna learn more about you and or Socket, where would you ask that they go on the internet? Yeah, I have a blog that doesn't get updated that much, but it has a bunch of my writing at Firas.org. And then you can learn about Socket at socket.dev.
And if folks want to email me, my email is just my first name at socket.dev. So feel free to hit me up. I always love talking about open source or security or whatever. So feel free to reach out. Well, that's fantastic. For Software Engineering Daily, this has been for us and Josh Goldberg. Cheers, y'all. Have a nice day. Thanks, Josh.