Menu Search
Jump to the content X X
Smashing Conf San Francisco

You know, we use ad-blockers as well. We gotta keep those servers running though. Did you know that we publish useful books and run friendly conferences — crafted for pros like yourself? E.g. our upcoming SmashingConf San Francisco, dedicated to smart front-end techniques and design patterns.

The Issue With Global Node Packages

Node.js brought about a great revolution for JavaScript developers by allowing us to write code that runs directly on our machines; our skills were no longer limited to browsers alone. At first, many of us simply saw this as a way to write our application servers without needing to learn another language, but we all quickly caught on to the fact that we could also write tools for the command line that automate a lot of things in our development cycles.

npm, which is bundled with Node.js, made this even easier by giving us quick and easy access to tools that others have created, which we install on our machines to access from wherever we are in our system. JavaScript was finally a “real” programming language. But with these new capabilities came a lot of best practices that needed to be discovered, because there were many new scenarios that wouldn’t be found in the browser. In particular, I’d like to discuss a practice that has been on my mind a lot lately that I think much of the community needs to evaluate.

What’s the Issue? Link

I’m specifically talking about installing packages from npm globally using npm install -g. Don’t get me wrong: Installing packages globally is certainly helpful and convenient at times, but we don’t always use it wisely.

Rule of thumb: If your project depends on a package, it should be listed in your package.json file as a dependency and installed locally in your project, rather than globally. Tools that your projects do not depend on can certainly be installed globally. For example, I use UglifyJS1 as a globally installed package to do one-off JavaScript file minification when the file isn’t part of a larger project or when I just want to share one file. Another good example would be the http-server2 package, which allows me to start up a simple file server in whatever directory I need with a simple command.

You might also be able to get away with using global packages if you’re working on an internal project, because many tools (like Docker) can use automation to neutralize some of the issues with global packages. If you’re working on a public and/or open-source project, though, please pay close attention because you are the primary audience of this post!

Why Shouldn’t I Install Dependencies Globally? Link

The obvious short answer is that your project depends on them. If your project depends on a package, it should be documented in package.json so that you can guarantee that it is installed when someone types npm install. Otherwise, you’ll need to add extra steps in your README file to inform anyone else who clones your project that they need to install each of your global dependencies as well.

For example, if your project relies on Browserify3 (we’ll be using Browserify in our examples from here on out), then you may have written a few steps in your README that look like this to help people get started with your project:

To use this project, follow these steps:

  1. git clone the repo.
  2. Run npm install.
  3. Run npm install -g browserify.
  4. Run browserify main.js > bundle.js to build.

Why force the user to add the extra step of installing Browserify globally? Besides making it simpler to guarantee that Browserify gets installed, adding it to your dependency list in package.json also guarantees that the correct version of Browserify will be installed. Having the wrong version of a dependency is often just as bad as not having the dependency installed at all. This means you should include the version of Browserify, and any other global packages you’re using, in your README file (I’m not sure I’ve ever seen anyone do this). This also means that if you update to a newer version of any of those packages, you’ll need to update the README with the new version as well.

Finally, even if someone installs the correct version of Browserify for your project, they may be working on a different project that requires a different version of that same tool, which would cause conflicts. Several of your own projects might even use different versions of Browserify because you updated it when you started a new project and didn’t go back to make sure that earlier projects were updated to work with the new version. These conflicts can be avoided.

What Can I Do About It? Link

The obvious answer is that you need to avoid using that -g flag when you install your packages and start using -S or --save to save them to your dependencies or -D or --save-dev to save them to your development dependencies. This, of course, isn’t the whole answer because it doesn’t explain how you can run packages like Browserify from the command line, which was the point of installing it globally in the first place. It wouldn’t be much of a solution if it couldn’t solve the original use case, would it?

Well, don’t worry. As I said, this isn’t the whole answer. So far, we’ve solved the issue of version collisions and eliminated a step and some maintenance from our README files. Before arriving at the best solution, we need to know an important fact: When you locally install a package that has “binary” files (i.e. it’s executable from the command line), then the binaries that are necessary for executing that tool will be stored in ./node_modules/.bin. This means you can use ./node_modules/.bin/browserify to execute a locally installed version of Browserify. Of course, nobody really wants to type all of that nonsense out, but it’s a start.

A quick fix would be to add ./node_modules/.bin to your PATH environment variable so that you can just run browserify to get it to work. At first, I was floored when I was informed that you could use relative paths like that in your PATH (thanks to a comment on another post I wrote4), but since then my emotions have leveled off because I realized that this only works when you are in the root directory of your project. The best workaround I could find is to throw a few more entries in your PATH so that you can do this from subdirectories as well (../node_modules/.bin/ and ../../node_modules/.bin/ and so on, for as many levels deep as you deem necessary); then, it should always be able to find the bin you’re looking for. Note that using relative PATHs has security risks, so use this only on your development machines.

Changing your PATH on your own machine is great because it saves you keystrokes, but I don’t think telling people who use your project that they need to alter their PATH is a great idea. One final solution takes a bit of configuration for each project but can be very helpful later on, especially for other users of your project: npm scripts5. In your package.json file, you can specify a scripts property that essentially creates aliases for your commands that npm can run. Let’s say your package.json looks like this:

{
    …
    "scripts": {
        "browserify": "browserify"
    }
    …
}

You could run npm run browserify, and it would run the Browserify version that you have installed locally to this project. The key for the property is the alias that you’re creating to use with npm run (for example, npm run $KEY), and the value is the command that will actually be executed. When you do this, npm will look for the browserify binary in the ./node_modules/.bin/ folder first before checking the rest of the folders in your PATH for it.

Of course, having to type npm run browserify instead of just browserify isn’t nearly as efficient, but I usually don’t use npm scripts like that. Instead, I set it up so that no one needs to know that I use Browserify, by creating a generic alias and letting it envelope a much larger command. For example:

{
    …
    "scripts": {
        "build": "browserify main.js > bundle.js"
    }
    …
}

Now, I can run npm run build, which lets everyone know that they are building the project, without telling them the nitty-gritty details of how it’s built, and I’m actually saving keystrokes. The encapsulation allows you to completely change the tooling and configuration of your build (making the change to webpack6, for example), without having to tell anyone about it or having to update the documentation.

npm scripts also allow you to pass other options to the command you are executing by first passing -- to tell npm that the rest of the parameters are to be passed to the command that’s being executed, rather than to be passed directly to npm run. For example, using the build script that I just created, we can run npm run build -- --debug, which would be the equivalent of running browserify main.js > bundle.js --debug.

npm scripts are very helpful tools for making common tasks easy to find and run and for giving other users really simple access to them. You could even go all out and learn how to use npm as your “build tool”7, which is becoming more popular. I put “build tool” in quotation marks because, technically, all it is doing is aliasing commands for your project, but people still tend to call it their build tool.

Augmenting your PATH and/or utilizing npm scripts might require a bit more work up front than just installing the tool globally, but I truly believe it is a better practice and will save us from some maintenance and compatibility issues in the long run, which is definitely a good thing. And you can’t go wrong by making things easier for consumers of your projects.

Can Or Should We Take This Further? Link

When you get right down to it, you come to realize that Node.js and npm are also dependencies of your project that can cause version conflicts. So, we should begin to list those as dependencies as well, somehow, to ensure that everyone can work with our projects with no fear of conflicts at all.

To do this, it is possible to install portable or local copies of Node.js and npm in your project. This has its own caveats. because you don’t want to store an installation of Node.js in your version control, and even if you did, that installation of Node.js likely wouldn’t work on a different computer because it would be specific to the operating system.

Also, this would require you to adjust your PATH to use the local Node.js and npm, and your users would be required to do so as well. These don’t seem like very good tradeoffs to me, so in my opinion, we haven’t reached the point where this is simple enough to do. Maybe some tools will emerge in the future to allow this, but we’ll just have to shove this idea to the wayside for now.

Conclusion Link

That’s all I have for you today. I hope you see the importance of keeping all of your dependencies listed and versioned in your project. If you agree with me, please help promote this idea by pointing it out whenever you see a project that doesn’t follow this principle. But remember to be nice! In fact, you could even consider adding a pull request to make the change for the project in question.

Excerpt image: npmjs.com8

(rb, al, ml, jb)

Footnotes Link

  1. 1 https://www.npmjs.com/package/uglify-js
  2. 2 https://www.npmjs.com/package/http-server
  3. 3 https://www.npmjs.com/package/browserify
  4. 4 http://www.joezimjs.com/javascript/no-more-global-npm-packages/#comment-1952307113
  5. 5 https://docs.npmjs.com/cli/run-script
  6. 6 http://webpack.github.io/
  7. 7 http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/
  8. 8 https://www.npmjs.com/package/http-server

↑ Back to top Tweet itShare on Facebook

Advertisement

Joseph Zimmerman is a God-fearing and loving family man, Web Developer for Footlocker.com, JavaScript Blogger, teacher, and amateur gamer.

  1. 1

    André Alçada Padez

    January 19, 2016 12:17 pm

    The article is good and informative. I’ve been following this practice since forever, relying on npm scripts, no additions to the path but i believe many people still don’t know this.
    It would be much more interesting if you would mention and explain how to stop depending on sudo to install and run global modules; it is dangerous, you’re basically giving the module to do anything he likes on the machine leaving it defenseless to atacks or bugs.
    If you want i can write you a quick explanation and tutorial later today, if you wish to post it as an update. Cheers

    14
  2. 4

    Jocelyn Lecomte

    January 19, 2016 1:52 pm

    I agree totally when you say that node and npm versions should also be locked. In our Java project, we use node because of the angular frontend.
    In the Java world, we usually use Maven to build the project, and the good news is that the frontend-maven-plugin (https://github.com/eirslett/frontend-maven-plugin) allows to lock node and npm version.
    This way, we can be pretty confident that the version of Node and NPM being run is the same in every build environment (to paraphrase the plugin readme :-) ).

    3
  3. 5

    Mustafa Enes Ertarhanacı

    January 19, 2016 10:21 pm

    You can actually specify Node or npm version dependency in your package.json with `engines` field which will produce a warning if conditions aren’t met.
    see: https://docs.npmjs.com/files/package.json#engines

    1
    • 6

      If you use nodenv, you can also use this plugin: https://github.com/hurrymaplelad/nodenv-package-json-engine

      Instead of choosing the node version based on .node-version file, it will use a node version matching `engines` from package.json.

      -1
    • 7

      Joe Zimmerman

      January 21, 2016 7:47 pm

      Yea, when I was talking about locking the Node and NPM version, though, I wasn’t talking about for packages that you install via `npm install` and/or depend on. I was talking about an entire project that you clone onto your numerous servers or allow others to clone onto their machines to run directly.

      0
  4. 8

    What do you think of nvm to deal with multiple node versions?

    1
    • 9

      Personally, I prefer [nodenv](https://github.com/OiNutter/nodenv) over nvm. nvm doesn’t support automatically switching versions based on the project.

      nodenv will automatically run the correct version of node based on the project directory. Put a `.node-version` file in the project root with the desired version of node, (file can be committed to git and shared with the team) and that version of node will automatically be the one used.

      -1
      • 10

        Joe Zimmerman

        January 21, 2016 7:49 pm

        That’s pretty awesome. Of course it only works if the users have nodenv installed, but it should at least work within a corporation. :)

        0
  5. 11

    +1 Down with global modules!

    As for the desire to lock node itself, I strongly recommend [nodenv](https://github.com/OiNutter/nodenv). Drop a `.node-version` file in the root of your project specifying which version of node it uses, and that’s the version that will be used! (Provided you have that version installed and configured for nodenv.)

    0
  6. 12

    Pawel Grzybek

    January 20, 2016 4:07 pm

    Ok. I thing I have never understood concept of global packages after reading this post. Can someone help me tp understand it?

    Lets say I have gulp installed locally on a project directory and another one installed globally. Lets say that local version is 3.9 and global i 3.8. After installing all dependencies on project by `npm i` and run `gulp` which version is actually running? Local or global?

    0
    • 13

      If you type gulp on the command line, you’ll be running the global version — version 3.8.

      If you add

      {

      “scripts”: {
      “gulp”: “gulp”
      }

      }

      to package.json and type npm run gulp on the command line, NPM will run the local version — version 3.9 — for you.

      The import part is that NPM runs the local version for you. It doesn’t muck with your global environment, just the environment it creates when you ask it to execute a script. NPM knows where the developer dependencies are installed, so it can do this safely for you (and you should let it).

      1
  7. 14

    There wasn’t anything about how to fix this, or remove stuff from the global scope. I found this though: https://docs.npmjs.com/getting-started/fixing-npm-permissions

    0
  8. 15

    Maybe npm could use the same technique as gradle wrapper with npmw.
    https://docs.gradle.org/current/userguide/gradle_wrapper.html

    0
  9. 16

    Bradley Farias

    January 22, 2016 4:07 pm

    npm bin can get you the current location of locally installed binaries.

    0
  10. 17

    I think the best paradigm for global packages is that they are command line sugar for the same library in your project.

    If the global package detects a package.json which has itself as a dependency it imports it’s core code from the version required by the project, otherwise it loads(requires) code from the global location.

    0
    • 18

      You’re right. Gulp will use the local version of Gulp to actually do the work, which is helpful to be able to use it without the need for npm scripts or adjusting your PATH, but if you’re not setting up the npm scripts, then others who grab your project still need to manually install it globally in order to use your build scripts.

      0
  11. 19

    Denny Christochowitz

    February 20, 2016 3:01 pm

    Nice post. Regarding the last point in your post: Locking down the version of Node.js and npm being used can also be achieved with containerizing your development environment via Docker. RisingStack recently made some well-versioned Docker images for Node.js available.

    3
  12. 20

    Hi Joseph,
    I am familiar with JavaScript, CSS, HTML5, JQuery etc. although I don’t work with it everyday spending 40 hrs/wk in a propriety ERP. I was a PHP/MySQL webmaster 10 years ago. I just read the article https://www.smashingmagazine.com/2013/11/introduction-to-full-stack-javascript/. I have bought a couple of books in the past but they start with all kinds of assumptions about previous knowledge. I was not a CS major but can type a few commands in a terminal window. What can you recommend as a ‘trail’ to learn ‘full-stack javascript’ that won’t trip you up at the very start?

    -1

↑ Back to top