Node.js powers an increasing number of applications in the modern web. As Node's popularity grew, npm evolved in parallel as its default package manager, and it has become a robust system for sharing and developing Node programs. And yet, npm is growing beyond its roots and is poised to become a generalized framework for all kinds of workflows within web development. Today's guest, Laurie Vos, is the CTO of npm, and he joins me on software engineering daily to explain how npm has evolved over time to incorporate more than its initial vision.
Laurie will be giving a talk at O'Gryley's Fluent Conference called, Beyond npm Install, The Many Uses of npm. Laurie Vos is the CTO of npm Incorporated. Laurie, welcome to software engineering daily. Hi, who's to be here?
Let's start with a naive question. What is npm? npm is the package manager for JavaScript. It started life as the package manager for the Node.js ecosystem, but in the last couple of years has become much more of a general tool.
I can go into what is a package manager, if you'd like? Sure, let's start there. So, one of the hardest problems in programming is code reuse. And one of the things that Node in particular is good at is code reuse through the use of many small modules.
Once you've adopted that pattern, though, you have a new problem, which is that your code in all of these small modules needs to be shared across projects, it needs to be shared across the world, it needs to be shared within your company. And once it's shared the first time, you need to keep it up to date. In the early days of Node, this was a huge problem. People were emailing each other justs and diffs, and saying, I go to this website and download this zip file and over to that website and download this tarball and glue them all together and make sure they don't conflict and blah, blah, blah, blah.
And making sure that your dependencies were in the right place up to date and not fighting with each other all the time is basically, those are the three things npm absolutely nails and has done since its beginnings. Great, and we'll go into each of those. The first one is that npm was created to improve the distribution of code. And you obviously mentioned the worst case distribution of code, which is like emailing each other tar balls and stuff.
But what are the different ways in which code gets distributed across the internet, perhaps that are a little more sane than things like that? Well, most modern languages or runtime environments have some sort of package management at this point. Pearl was the first out of the gate with CPAN, which was a repository. It was one half of the package manager.
It was where all the packages got uploaded, but it didn't really have a dedicated client to download them. You just sort of downloaded and installed the stuff into your project. PHP in its early days had pair. Obviously, all the Linux distros have had package managers since forever, red package manager and yum and apps and all of those.
They're all obviously Ruby gems. They're all strikes at the same problem. A lot of them have problems that are sort of intrinsic to the environments in which they run a lot of the advantage of using npm or the convenience of using npm. The fact that you never get into this problem referred to as dependency hell.
You never get into the situation where a requires b and c requires a different version of b and the two of them can never meet and they can never use a and c at the same time. That doesn't happen in node. And the reason that doesn't happen in node is because node is written specifically to make that impossible node and its package manager were written at the same time by the same people, which is a completely unique feature of a runtime environment. No other runtime environment was written with the expectation that there would be a package manager and therefore you should make it easy to manage packages.
Right. So you've touched on this idea of dependency hell and you've defined it. Why was dependency hell so prevalent in the JavaScript world of development? It's I'm not sure that's the correct premise of the question.
It is why isn't it so prevalent in the world of JavaScript because it's prevalent everywhere. You get dependency hell in all runtime environments. You don't get it in node because node was built to avoid it. It was built with the expectation that this would happen.
And as a result node is the reason node is a more modular landscape. The reason that many small modules as a philosophy caught on so much better in node is because you could get away with it. In other languages, that's been everyone's always known that using lots of small components is going to be better. But in languages like C or PHP or Perl or Java, having lots of libraries is really painful.
Installing any particular library is always going to be this huge ceremony to make sure that it's not contacting with anything else. And so they tend towards monolithic libraries like two or three per project just because two or three is about as many as you can stand the pain of installing. Whereas in NPM and node, its routines absolutely expected that a big project will have a couple thousand dependencies, which is. And the reason that happened is because there is no dependency hell.
So you touched on the fact that this was possible largely because NPM was developed at the same time that node was developed. So I'm kind of curious about how the communication process worked across that development period of time. So I should definitely include the huge caveat that I wasn't there at the time. I was not the person who was building it.
Isaac, my CEO, is the inventor of NPM who put this together. But I've been in the room when he explains history long enough that I can say that at the time it was a very easy way to do that communication just because there were only about 10 people in the world who knew about this thing. So Isaac was writing NPM and Ryan Dahl who invented node. They were good friends and they spoke every day and they lived in the same city and they could meet each other.
And that was how that happened. It was collaborative by default. So how is this, let's say what were the things that node and NPM were able to, were there any silver bullets that node and NPM were able to, were there any silver bullets that node and NPM being built in parallel were able to improve upon when you think about NPM relative to other package managers like App or RPM or whatever, the other canonical package managers? Absolutely.
I've touched on it already, which is the reason dependency hell is impossible in node is because node was written with the expectation that NPM would be that it putting packages into a certain place. So node modules directory and package JSON are actually expected by node. The thing that defines the node module is part of NPM and it is also part of node in a way that is completely unique to that environment. So literally node pulls stuff into memory and it's unique features that it can pull two or more versions of the same library into memory at the same time and run them at the same time without conflicting with each other.
And it does that because it has the expectation that NPM will have put them on disk in a pattern that it can predict such that it can load them into the right order such that the right one will be getting the right piece of code without conflicting with each other or without running version one when you were expecting version two. And I was going to say it's a relatively simple, it's an extremely complicated piece of code, but it's a relatively simple concept to get that to happen, but it's only because they were written, node was invented after package manager that they can't copy it or that it was an idea that somebody should do that. Great. Okay.
So I want to get into a more granular level of the topics that you're going to be exploring in your Fluent 2016 talk, and the talk is called the Mini Uses of NPM. So to begin with, how do people use NPM commands to get started with a project? So the thesis of the talk is that maybe 90% of NPM users know about NPM install, and that's all they know about NPM. They just run NPM installed, they know that it brings down a library and puts a library in place, and they're unaware that there are any other features of NPM whatsoever, which is both a good and a bad thing.
It's great that we build a tool that doesn't require you to know anything other than one command to be an extremely useful tool, but it's also a problem for a team of people who spend their whole lives making sure that all the other things NPM does work, if only if 90% of our users aren't aware of them. So the purpose of the talk or thesis of the talk is that NPM is not just a tool that you use to kick off your project, it is a tool that you use at every point of the life cycle of your project. So there's commands, as the most basic one, there's NPM in it, which starts your project. A lot of people start an NPM project by manually hacking a package JSON together, and that's just what NPM in it will do for you.
Just ask you a bunch of questions and do it for you. But there's NPM publish will put stuff on the library on the registry for you, but there's also stuff like NPM run scripts, where you can do stuff like NPM test, NPM starts NPM run any random word that you like, and you can build in life cycle chunks of your code. So if you have a command that you run all the time that resets your test fixtures, or sets the testing database back to zero, or any other ordinary development tasks that you need to share between all of these developers on your team, and you would ordinarily write a wiki page saying, when you need to fix this problem, run this command, or when you need to do this thing, right, this command, you can just build it into the application itself. The application knows how to take care of its own life cycle, because you can just build that into package JSON, and thus into NPM itself.
Okay, great. So what kinds of NPM commands are useful for improving a testing and deployment workflow? Or how should people architect their testing and deployment workflows around NPM? That is a deep and complicated question, and I do not think it would be wrong for me to say that there's one answer to that question.
I cannot possibly say this is the way you do testing with NPM, because even within NPM, I would be shouted down from all sides, where I would pick one way. The crucial advantage of NPM is that you can use NPM's life cycle hooks. It has things like pre-publish and post-install and pre-version scripts that you can hook in, and they will happen at life cycle events. So you can make it such that it is impossible to publish a new version of your package, unless all of the tests pass.
You can just build that into the workflow, and it just becomes part of everyone's life that they know that this happens. One of the things that we've done all the time is we put linters into the pre-publish hooks of our packages. So every time you publish the package, it reformats the code of the package such that it meets the NPM style guide, or one of the NPM style guides we have. Everybody gets to pick their own.
But it had this really fantastic effect that it meant that code reviews turned into a completely different animal, because there was no longer this sort of lazy code review where people would go, you've put a semicolon in the wrong place, or I don't like this formatting, or this variable isn't used, or your callback might not get called, because it's literally impossible to publish a version of our packages that has any of those problems. So instead, our code reviews had to be much deeper and more complicated things, where you were like, well, this approach looks interesting, but there could be a more important way. And that's basically what NPM does across the board, is NPM reduces the amount of communication that is required to do ordinary things. You no longer have to tell somebody the command, you no longer have to remind them to run their tests, you no longer have to give them elementary code reviews, you can automate all of that stuff away and get on with the complicated parts of programming.
That's what NPM is trying to do. So the assumption that people make about NPM that you kind of said a few minutes ago was that you only use NPM to install stuff. And it's too bad that people make that assumption, but at the same time, it's kind of a fair assumption to make, because the way that we have historically used package managers, or at least the way that I've used package managers in the past, is generally for that purpose. So how can we start shifting, like as a developer, how can I shift my paradigm to thinking of NPM in the more the wider scope that you, for example, see it?
That's an excellent question. And I think the answer to that question is not simple, and it's sort of crucial to the future successes. Is it a question for the marketing department? It's a question for the person who writes all of our documentation, for sure.
But it's crucial to the success of both the NPM ecosystem and NPM Inc as a company that people become aware of the wider universe. I think there's a diagram that I draw that's part of the talk, it starts to install, and it's a series of chains of lifecycles. This is you going off on the install flow, this is you going off on the publish flow, this is you going off on the deploy flow. And the deploy flow is like this series of question marks, because there's a couple of things that NPM sort of does, but not perfectly, where it gets dicey, like shrink wrap.
Shrink wrap's a really good idea in theory. The idea of shrink wrap is that you go in development, I want to use semantic version, in development. I always want to bring in all of the patch versions because they're going to be safe. I usually want to bring in the future versions because you features are fun, and the developers promise that nothing is going to go wrong.
But in production, you probably don't want to just work on promises. In production, you want to be like, I want the exact snapshot of what is running right now and working on my development environment in production, and that's all that I want. That's what shrink wrap is supposed to do. And it does it most of the time, but it definitely has some edge cases where it breaks in a really painful way.
And that means that we can't just blanket say, and you deploy your stuff with shrink wrap, because we know that you can't sometimes deploy your stuff with shrink wrap. We know it will break sometimes. So that's one of the things that we are focusing on for NPM in 2016. We have a big list of them, but one of them is NPM works every time, and it works the same every time.
Because we want it to be completely predictable, completely understandable, totally unsurprising, such that you're not scared of using NPM to run your entire workflow. Because one of the things we said very early on is that NPM exists to reduce developer friction. That's why people like this is we just make it easier to get stuff done. And if we're not completely reliable, then we are introducing friction.
We're not taking it away. So it's a focus for us. So just to go further into this question, before I started researching for this interview, I really just thought that NPM was just a package manager, but it seems like the vision for NPM is so much bigger. It's more like bash scripting or chef.
And so I want to make sure that I'm correct about this being the current vision. And if so, I'm curious if this was the vision from the beginning. It's not, I would say saying that it's like bash scripting or chef is an over each. We are, the NPM run scripts are written in JavaScript or bash or whatever you were running them in before.
If you wanted it to run your chef scripts, it could. NPM is much more like a framework for a workflow. It's not defining the pieces of the workflow. It's not saying this is how you run your JavaScript or it's not saying this is how you should make a website or this is how your API should look.
We're not, we're not rails. We're not prescriptive. What we are is we are aware that software works this way. You start a project.
You install dependencies. You run it. You test it. You deploy it.
You fix it. You add features. You occasionally do breaking changes. And we know how to do all of those things.
And however it is that you do all of those things, we know that you have to chain them together in the same way. And we will get out of your way. One of the tools that we came across recently that we think is fascinating is this company called Greenkeeper. And what they do is they hook into the NPM changes fee.
So they watch every single package being published. And if you tell them what packages you own, they will tell you when your dependencies change. They will say these three packages that you depend on have changed. Now you know.
And you can either pull in those changes or update your packages or change production. It will tell you the change log. This is a security fix. You probably really want to do this one.
And that's exactly the kind of use case that we like for NPM. Where we are just a mechanism by which people are reducing communication. You don't have to go to those libraries and read their web pages to discover that they had a security alert. It just happens automatically.
Just get an email saying, hey, time to hit the update button because this one's scary. That is the kind of reduction in friction and reducing of human communication that we really think adds a bunch of value. And it's what we like. We're developers.
We want to just be writing code all the time. Not having meetings. And NPM is really good at that. So I think one example that describes this goal that you just described is that NPM can be used to manage the workflows of different teams.
So if I have three teams, like a web team and an API team and a QA team, how could I use NPM to improve the workflow between these teams? Well, we have a very concrete example because NPM does have a web team and an API team and a support team. And they do communicate via the medium of NPM. We have modules within the company that do stuff that are common to the various bits of NPM.
NPM is not it sells a monolith. NPM, the service, is a series of disconnected microservices which are talking to each other all the time. So we know when team registry publishes a new version of the module which validates what a package should look like, that the CLI team can pull in the new version of that module and go, okay, well, now the CLI should be validating the same thing that the service can validate because otherwise it will be possible for the CLI to be a time to publish something that the server will then reject. But those teams don't need to have meetings about that.
That just happens. And in the same way the team that works on the website can change how user names are validated or spell checked or whatever it is. And that stuff turns out in the registry site as well. So does that answer your question?
Yes, it does. And just to give one other example of how people can use NPM before we get into how NPM works at a lower level, some people use NPM to actually manage CSS. And I think this is maybe part of a discussion we could have later on about what is the big vision for NPM. But explain how people are using NPM to manage their CSS.
So I said at the beginning, people, we are no longer the package manager for Node. We are the package manager for JavaScript. And that was not like some crazy meeting we had. We were like, let's wait for the fences and do all the JavaScript.
It was very much an NPM-y sort of get out of their way type thing. People were using us for front-end code whether we liked it or not. It was just happening. And so we decided to get the hell out of their way.
And CSS was one of the ways they were doing that. There is a project, a project we quite like, called Dr. Frankenstein, which it was being used at Pivotal, I think. And it basically modularizes your CSS.
You can write, if you have a very large CSS component library that is officially put together, you can bring in your components independently. But on any individual project, if you have lots and lots of web projects and you are using the same tools, so on this tiny little mobile site you are only using two components. But on this big CMS site you are using 50 components. So whichever components you are using, you can just NPM install that chunk of modular CSS and it works.
So that is one of the ways it can be done. There are a bunch of third-party modules that will let you NPM install just chunks of CSS. In some cases, chunks of HTML and JavaScript as well. I think it is like a calendar widget or something that is canonical with its people and bent over and over.
We just install the widget and it is done and suddenly that widget is available in your web app. And the use of NPM to be the way you build websites is something we absolutely are committed to doing. We don't want it to be the way you build websites and the way that rails with the way you build websites, like structure your code this way. We want it to be the default tool for web developers.
Like we want the first thing you do when you are like, all right, I am going to build a website today to be NPM and whatever it is. We want to be the way you do that. We want to be the framework on which all of those tools hang. Just because that is the thing that we have seen that is useful.
Can you delineate that further? So how would that differ from what the rails CLI tool would do? So the rails CLI tool is a tool that lets you build a rails application. It is expecting a model view controller application with your template here and CSS here and your views here.
NPM is a much smaller concern. NPM is just if you wanted to start your projects, this is how you would install your dependencies. If you wanted to run it, if you wanted to test it, if you wanted to bring in other dependencies working with each other, you would not necessarily say, this is the way you build your website. That is a change for me personally.
My background is as a web developer, when I came in to NPM two years ago, I was like, all right, we are going to find the best web framework and pick it and pick it. This is how it will go. I felt like for good six months, the thing about Node that makes it good, the thing about Node that makes it different from Ruby is that nobody wants to pick a winner. In fact, that is its strength.
There is no one way. Nobody says this is how you do it. There is probably going to be less than there are now. If you want to build a website, there is like, here is 10,000 frameworks pick one.
It is a Cambrian explosion. I think there is going to be a lot of consolidation. There is going to be a lot of merging of projects or at least cherry-picking of the best features. I think we are going to shake out with somewhere between five and ten major use cases.
The pure API play, the full-on WordPress-esque website where you just install individual chunks of functionality, but it always works the same. The middle ground where it is like expressing. It is a feature of a web app, but it does not say what the web app should look like. A couple other points on that spectrum.
There is sales which is exactly like Rails which is a point on that spectrum. Sales is not going to be the thing that wins. Nothing is going to win. It is going to be this set of tools that cover the major use cases for building something that is on the web.
I think that is fantastic. It is much closer to the reality of how people build web apps. This reminds me of the conversations I have had with people from the Facebook team. They are working on projects like React and Relay and Flow.
I remember asking, I think it was Ben. I forgot his last name. It has been something that will be in the show notes. Why is not there some sort of one React tool to bind them all to give me this wonderful package experience that tells me how to do everything in React?
It was very clear from the conversation that they want it to be loosely coupled and they want people to be able to pick from things like it is a buffet. You have this set of dishes that you are going to be served. You have to choose what you want and hopefully they work together in a nice convenient way that is as expected. Let's talk about how npm works.
Just from a basic level. When I use npm to install a module, what happens? The most important thing is that it hits a bunch of servers that we run that have all the TARBOLs on it. One of the most surprising things about working at npm is your discovery that there are huge tons of npm users who don't understand that there is a registry there at all.
There is a huge chunk of users who think what npm does is it downloads your dependencies from GitHub for you. This leads to some fantastic support conversations where people are like, how did the new thing to GitHub but it is not on npm? What is wrong with you? Why are you broken?
We say you missed the step where you put it on our servers. We frequently get this complaint from two directions. One is, why are you running a registry server? You should download everything from GitHub.
We were like, well, GitHub goes down sometimes. They are pretty good at their jobs. Other times we get it the other way. Why would you run a bunch of servers that could go down?
It is true. It is a lot of things that are famously used to be down all the time. The answer is it is much faster and more convenient if it is on servers that are set up in the way that npm is expecting so it works the way the client works and can do all the complex version and stuff the client can do. To get back to your question, the lifecycle is quite important because it is a thing people don't understand when they see npm hitting the network all the time.
They are like, why does it download everything every single time? It hits the network all the time but it doesn't download everything every time. What it does is for every single package it asks the registry, is there a new version of the package? If there is a new version of the package, it loads it from the cache.
There is a local cache on every computer that has npm installed which is all of the modules you have already installed. One of the reasons it looks like it is hitting the network all the time is because it is making three or four requests. Making these very quick library requests which is is there anything new? The answer is no.
It moves on to the next package. But also when you have a thousand dependencies, the chances are between running npm install one time and a week later, maybe 50 of those will update. It does have new steps to download all the time. If you are following a number and you are allowing patch versions coming all the time, 50 patch versions out of a thousand packages is a good percentage of the ecosystem.
It is downloading new stuff all the time because new stuff has happened. That is the first part of the lifecycle. It is a chain of three or fours and 200s. It pulls down either notices that nothing has changed or stuff that has actually changed.
It unpacks those to a temporary directory. It makes sure the tree is the way it needs it to be. It puts it into your node module directory as a single atomic step. I am radically simplifying.
When you mention the global cache, what is the interaction between the global cache on my machine and the different applications I have built on my machine that use packages in the cache? The cache is used at install time and only at install time. For any individual node application, your node module territory is where all the packages you are using at that specific time live. The advantage of this is that if you are in project A and you install a new version, you cannot accidentally install a new version that makes project brick.
Project B install all of this modules and it was fine. It is still fine. Project A, which you ran a week later, had a bunch of new dependencies and they still work fine as well. Everybody's local node modules copy is completely independent of each other.
The only thing the global cache is used to do is reduce the amount of stuff that you download when you run npm install. If you are running npm install and it is like you installed this last week and 99% of these packages are unchanged, it is all going to come from one portion of your disk to another portion of your disk and be super fast. The npm nomenclature for where the packages live remotely is registry. Is there an actual difference between the term registry and repository or is it more just like a contextual thing?
It depends on how much of a pedantic nerd you feel like being. I am a repository. A repository is a monolith. Everything has been crammed into here and a Git repository is all connected to each other.
The registry is an index. It is not saying this is the current thing. Here you go. Here is a list of all of the things.
It is relevant to you. Most of the time when you are using npm, what you are doing is taking a slice of the registry. Every registry package has a set of tags associated with it and the latest tag is where everybody lives. When you say npm install, the latest version of everything.
That is not what the registry contains. Every version of everything. It contains all the versions you have published. If you want to go give me the pre-release versions of everything.
Give me everything that is not deprecated. Give me everything that is currently in beta but is marked as being stable. Give me nightly builds. Depending on the sophistication of the authors, those can be meaningful tags.
As we have discussed, node avoids dependency hell with semantic versioning. I will not ask you to go into extreme detail about how semantic versioning works. Are there any tidbits or elements of semantic versioning that developers should know about? That will be helpful when they are debugging some sort of issue.
What is useful for a developer to know about semver? I think the most interesting, interesting simple but incredibly useful thing, I cannot remember who threw it at me recently. It was a Gdub to write documentation. We always talk about semver as being the three members separated by dots.
We say major, minor, and patch. To understand semver, all you have to do is change those words. Instead of major, major, and break, use breaking feature and fix. That is what the numbers are.
If the first number changes, it is a breaking change. If the second number changes, it is a feature. It is added a feature but it is not broken. If the third number changes, no functionality is changed.
There is something fixed. If you allow the third numbers to change, all you are going to get is the current functionality slowly being more perfect. If you allow the second two numbers to change, you will get new features occasionally. If you allow the first number to change, it is because you are expecting to have to do some work.
You are deliberately bringing in a breaking change. You know that something in your software is going to have to be upgraded. You have decided that is okay. I want to zoom out and talk about the bigger picture and perhaps some common misconceptions.
We have done a number of shows about JavaScript technologies including node. You have pointed out the difference between JavaScript and node. You have emphasized in a talk that I saw that node is a runtime. Why is this distinction important?
What does this term mean? It is one of the trickier semantic problems of living in node land. People think about node as being a language. JavaScript is the language.
JavaScript running in the browser and in the server, they share a syntax but there are important differences between them. The only way to describe them is the runtime environment in which JavaScript is different. The biggest and most crucial difference is I have repeatedly said that node doesn't have dependency hell. It does not have dependency hell.
JavaScript can have dependency hell. If you import two libraries in your browser and fight with each other, they will fight with each other and break. There is nothing that anyone can do about that. That is just how browsers work.
That is why when we are saying that node is the runtime and node has this great teacher. We are limiting the scope of our claim of magicality. We will solve this problem for you. We can only solve it for you on the server.
We cannot fix how modules work on the browser. ES 2050 and ES 6, the new versions of JavaScript which no one can agree on a name to call. They have a module syntax and semantics about how this should work. There is still a fair amount of disagreement about how they should work exactly.
The most obvious evidence of that being that if you ask anybody if there is disagreement, they will not know. They will not have the same thing. Everyone agrees. If you ask everybody, everybody agrees why.
They don't even know they disagree. To reiterate our discussion from the beginning, why does JavaScript need a package manager? JavaScript needs a package manager for all the reasons that node need a package manager. Web development as an industry as a profession is a relatively young one.
It is beginning to mature and people are beginning to realize it may be reinventing my entire web app. That was what that problem was brilliantly solved by Rails on the server side. I am not a big fan of Ruby on Rails, one of the things it did was introduced to web developers and web development as a profession. All you need is a pattern.
It doesn't need to be the best pattern. It doesn't need to be a fantastic pattern that fixes every problem. People can follow and use their default. Then suddenly everyone is more productive.
You hire an engineer and you go, it is a Rails app. Suddenly the first six weeks of their life their company are gone. You don't need to train them on how their template system works and how their model controller works because it is a Rails app. It works on a Rails app before.
The reason you need a package manager for frontend JavaScript is because that is going to happen. That is already happening for the complicated client side single page apps that people are building in huge numbers these days. There is Ember, Angular, there is eight zillion other frameworks. They have tons of components and they are very modular.
You need to share them between each other. NPM cannot fix dependency but it can tell you where your modules are. Put them into disk in a predictable way. We can solve that problem for the browser.
How does the world change if we get a package manager for HTML and CSS along with this package manager for JavaScript? I think if we do it right, it will be the same world but faster. What NPM is doing for the server is people talk about how much faster node development is and it is not that node itself runs faster. It is a performance language but it is not high performance.
What is faster about working in node is the team. The team works faster in node because there is less friction and they don't spend an hour a day installing dependencies or fighting with each other about what the run script is. Because all of that stuff is extracted away or automated away by NPM, no team runs faster. There is a case study used from PayPal and it is a brilliant case study because they literally had two teams building the same thing.
They were like we are not so sure about this node thing and we are PayPal so we can throw two teams at exactly the same problem at the same time and the node version of that project was finished in half the time using half the developers and used a quarter of the hardware. It was better on every metric. That is what you can do for the front end. It is not that node is a super fast run time.
It is that having package management makes your team work together more smoothly and that makes your development process faster. So bringing that to the front end is exactly as you saw it is bringing it back in. It is worth pointing out here that NPM is both a package manager and a company. What is the company NPM which you are the CTO of?
What does NPM do? Well, that is a complicated question. First and foremost, the reason we formed a company, the problem statement Isaac came to me with when he was like I think I need to turn this thing into a company is the registry is too big to survive on the generosity of its users. We know from our conversations with the other people who run package sprint repositories that running it based on donations is a nightmare hellscape that you don't want to get into.
There is too many people, too much bandwidth, too many servers. The purpose of NPM Inc is to make NPM run forever. That is what it is supposed to do. The way you do that is you wrap a business all around it.
You go this is the precious egg that is NPM and to keep it going we have to cushion it in a pile of money. We could just beg our users for money. That happened a couple of times in the early days of NPM but it was not a sustainable and created weird tensions. That is what NPM Inc does.
It adds one feature to the registry and that features privacy. If you want to share your code with the world, that is free. If you want to share your code with just the people inside your company, that is the bid you pay for. That is what private packages is.
That is what NPM onsite are. They are just different versions of how you want to share stuff privately. I think these combination open source plus business model companies are super interesting. I have done interviews with people from a lot of these companies like Confluent, Kafka, and Cloud-era people.
It is interesting. Could you tell me about your work as CTO of NPM? First I would like to draw the distinction between us and some other open source companies. There is a pattern of running an open source company which is we build this server and we run this server for you.
You can also run this server for yourself in exactly the same way. That is true. You could run your own NPM repository but we are different in that we are also the registry. A huge chunk of why NPM is valuable to the world.
It is valuable to its users because we are literally the world's largest repository of software of any kind at this point. We have so much more than anybody else. You could run your own NPM server but it is not really a use unless you have all the modules. That is what we are doing is taking care of the modules for everyone and making sure it is not full of junk.
That is a big part of what we do. You had a more specific question which is what do I do as the CTO. That is a more complicated question because that answer is not stayed the same for more than about three months at a time. For the first three months I was doing nothing but fighting operational fires because the registry was down all the time.
For the next three months I was writing the next version of the registry. For the next three months I was drawing architecture diagrams because we hired a bunch of people and they needed to write stuff. I just needed to say what it looked like. For the next three months I was just hiring.
For the next three months I was just managing people. For the next three months I can't remember what saved the company we were at. Now what I do. The last three months have been defining products because there is now enough of us going on and there is enough people writing software and people managing the software.
The thing we discovered was the thing most needed doing was saying what is it and what does it do. Importantly what doesn't it do. We have so many smart people coming at the problem from every direction saying we are not going to do that yet is a big part of what we are doing as a company is staying focused on. This is the most useful thing we should do.
Fascinating. It is how you have been. At least you don't get bored. It is the thing about being a founder.
The average CTO would say my job changes every three months. The founder does whatever the hell needs to be doing. The CTO title is phenomenal for me. Final question.
Where do you see web development going more generally? Do you have any hyper-ambitious counter-intuitive things I probably haven't heard before that you can tell me about your vision for where web development is going? I think if it was something you have never heard before then it would be almost certainly something that is not true. I am not some visionary genius of web development because ten years from now we are going to be reacting on the server.
Who the hell knows? I think the thing that is already true that not a lot of people have fully embedded into their consciousness is the dominance of mobile. I think a lot of people who are web developers, especially people who have web developers as long as I have, they still think of mobile as that thing that would be nice to have but really want to build things, a thing you look at at a laptop. That is demonstrably not true.
50% of all web traffic in the US is on mobile. That is on phones outside the US. It is 70 or 80% depending where you are. The web is a thing we experience on screens that are four inches wide.
The effects that has, the ramifications that has for the stuff we should be building and how fast it can be and how complicated it can be, we are not thinking about that enough right now. When people architect their apps, maybe they think a little bit too much about the front end and think about the mobile as an afterthought. That is an anti-pattern. Absolutely.
I think in particular it is a problem for people who are building single page apps. One of the things about single page apps is they use a lot of processor and a lot more bandwidth. People are like, that is fine. Connections and megafast and supercomputer.
That is true. This thing happened where people stopped using laptops. Half of your audience are on this tiny crappy computer. The fact that it does not matter on desktop, you are right.
It does not matter on desktop. It does not matter. What does it work like on a phone? Your constraints are primarily those of a phone.
How it works on a desktop is just nice to have. That seems like a great place to close off. Laurie, thanks for making time to come on to software engineering daily. I look forward to seeing you at the Fluent Conference and I will have a t-shirt to give you for sure.
We will have fun to get back to you. Oh, thank you. Wonderful.