JSJ 266 NPM 5.0 with Rebecca Turner

On today’s episode of JavaScript Jabber, Charles Max Wood and panelist Joe Eames chat with Rebecca Turner, tech lead for https://www.npmjs.com/, a popular Javascript package manager with the worlds largest software registry. Learn about the newly released NPM 5 including a few of the updated features. Stay tuned![1:58] Was the release of node JS 8 tied to NPM5?- Features in NPM5 have been in planning for 2 years now.- Planned on getting it out earlier this year.- Node 8 was coming out and got pushed out a month.- Putting NPM5 into Node 8 became doable.- Pushed really hard to get NPM5 into https://nodejs.org/en/blog/release/v8.0.0/ so that users would get NPM5 and updates to NPM5.[2:58] Why would it matter? NPM doesn’t care right?- Right you can use NPM5 with any version of node.- Most people don’t update NPM, but upgrade Node.- So releasing them together allowed for when people updated Node they would get NPM 5.[3:29] How does the upgrade process work if you’re using NVM or some node version manager?- Depends. Different approaches for each- NVM gets a fresh copy of Node with new globals. NVM5 and Node 8 are bundled.- For some, If you manually upgrade NVM you’ll always have to manually. It will keep the one you manually upgraded to.[4:16] Why NPM 5?- It’s night and day faster.- 3 to 5 times speed up is not uncommon.- Most package managers are slow.- NPM 5 is still growing. Will get even faster.[5:18] How did you make it faster?- The NPM’s cache is old. It’s very slow. Appalling slow.- Rewrote cache- Saw huge performance gains[5:49] What is the function of the cache?- Cache makes it so you don’t have to reinstall modules from the internet.- It has registry information too.- It will now obey http headers for timing out cache.[6:50] Other things that made it faster?- Had a log file for a long time. It was called https://docs.npmjs.com/cli/shrinkwrap.- NPM 5 makes it default.- Renamed it to packagelog.json- Exactly like shrinkwrap package file seen before- In combo with cache, it makes it really fast.- Stores information about what the tree should look like and it’s general structure.- It doesn’t have to go back and learn versions of packages.[7:50] Can you turn the default Packagelog.json off?- Yes. Just:- Set packagelog=false in the npmrc[8:01] Why make it default? Why wasn’t it default before?- It Didn’t have it before. Shrinkwrap was added as a separate project enfolded in NPM and wasn’t core to the design of NPM.- Most people would now benefit from it. Not many scenarios where you wouldn’t want one.- Teams not using the same tools causes headaches and issues.[9:38] Where does not having a lock show up as a problem?- It records the versions of the packages installed and where NPM put them so that when you clone a project down you will have exactly the same versions across machines.- Collaborators have the exact same version.- Protects from issues after people introduce changes and patch releases.- NPM being faster is just a bonus.- Store the sha512 of the package that was installed in the glock file so that we can verify it when you install. It’s Bit for bit what you had previously.[11:12] Could you solve that by setting the package version as the same version as the .Json file?- No. That will lock down the versions of the modules that you install personally, not the dependancies, or transitive dependancies.- Package log allows you to look into the head of the installer. This is what the install looks like.[12:16] Defaulting the log file speed things up? How?- It doesn’t have to figure out dependences or the tree which makes it faster.- Shrinkwrap command is still there, it renames it to shrinkwrap but shrinkwrap cannot be published.- For application level things or big libraries, using shrinkwrap to lock down versions is popular.[13:42] You’ve Adopted specifications in a ROC process. When did you guys do that?- Did it in January- Have been using them internally for years. Inviting people into the process.- Specifications- Written in the form of “Here is the problem and here are the solutions.”- Spec folder in NPM docs, things being added to that as they specify how things work.- Spec tests have been great.[14:59] The update adds new tools. Will there be new things in registry as well?- Yes.- Information about a package from registry, it returns document that has info about every version and package json data and full readme for every version.- It gets very large.- New API to request smaller version of that document.- Reduces bandwidth, lower download size, makes it substantially faster.- Used to be hashed with sha1, With this update it will be hashed with sha512 as well as sha1 for older clients.[16:20] Will you be stopping support for older versions?- LTS version of NPM was a thing for a while. They stopped doing that.- Two models, people either use whatever version came with Node or they update to the latest.- The NPM team is really small. Hard to maintain old NPM branches.- Supports current versions and that’s pretty much it.- If there are big problems they will fix old versions. Patches , etc.[17:36] Will there ever be problems with that?- Older versions should continue to work. Shouldn’t break any of that.- Can’t upgrade from 0.8.- It does break with different Node version- Does not support Node versions 0.10 or 0.12.[18:47] How do you upgrade to NPM?- sudo npm install -gmpm- Yes, you may not need sudo. depend on what you’re on.[19:07] How long has it been since version 4?- Last October is when it came out.[19:24] Do you already have plans for version 6?- Yes!- More releases than before coming up.- Finally deprecating old features that are only used in a few packages out of the whole registry.- Running tests on getting rid of things.[20:50] Self healing cache. What is it and why do we want it?- Users are sometimes showing up where installs are broken and tarbols are corrupted.- This happens sometimes with complicated containerization setups makes it more likely. It’s unclear where the problem actually is.- https://www.npmjs.com/package/cacache - content addressable cache. Take the hash of your package and use it to look up address to look it up in the cache.- Compares the Tarbol using an address to look it up in the cache.- Compares to see if it’s old. Trashes old and downloads updated one.- Came out with the cache. Free side effect of the new cache.[23:14] New information output as part of the update?- NPM has always gave back you the tree from what you just installed.- Now, trees can be larger and displaying that much information is not useful.- User patch - gives you specifically what you asked for.- Information it shows will be something like: “I installed 50 items, updated 7, deleted 2.”[24:23] Did you personally put that together?- Yes, threw it together and then got feedback from users and went with it.- Often unplanned features will get made and will be thrown out to get feedback.- Another new things ls output now shows you modules that were deduped. Shows logical tree and it’s relationships and what was deduped.[25:27] You came up to node 4 syntax. Why not go to node 8?- To allow people with just node 4 be able to use NPM.- Many projects still run Node 4. Once a project has been deployed, people generally don’t touch it.[26:20] Other new features? What about the File Specifier?- File specifier is new. File paths can be in package json, usually put inside pointing to something inside your package.- It will copy from there to your node modules.- Just a node module symlink.- Much faster. Verifiable that what’s in your node modules matches the source. If it’s pointing at the right place it’s correct. If not, then it’s not.- Earlier, sometimes it was hard to tell.[27:38] Anything else as part of the NPM 5 release? Who do you think will be most affected by it?- For the most part, people notice three things:- 1st. no giant tree at the end- 2nd. Much faster- 3rd. Package lock.[28:14] If it’s locked, how do you update it?- Run npm installer and then npm update- Used to be scary, but works well now.- Updates to latest semver, matches semver to package json to all node modules.- Updates package lock at the same time- Summary in Git shows what’s changed.[28:59] Did Yarn come into play with your decisions with this release?- The plans have been in play for a long time for this update.- https://yarnpkg.com/en/ inclusion of similar features and the feedback was an indicator that some of the features were valuable.[29:53] Other plans to incorporate features similar to yarn?- Features are already pretty close.- There are other alternative package managers out there.- PMPM interesting because when it installs it doesn’t copy all the files. It cBecome a supporter of this podcast: https://www.spreaker.com/podcast/javascript-jabber--6102064/support.