Open Source – cave lupum


I’ve been actively working with open-source JavaScript packages for about 18 months. Developers that are very generous with their time have built tools and frameworks that have enriched the lives of developers all over the world. I too have contributed tools and believe in this beautiful Ecosystem.

A few months ago I started to look under the hood of my SPA and Nodejs applications and found code and practices that caught my attention. I found packages that other packages depended on, have very few lines of code. Packages with dependencies that are out of date or dependencies that had warnings such as, this package version is subject to a denial of service attack.

Upon further reflection, I got very concerned about the damage a bad person could inflict on trusting developers that download packages that have a dependency that has been replaced by evil code. My system and software that I write could be compromised. Now imagine ticking time bomb code replicated over Docker Containers and placed on servers. Damage could be immeasurable.

cave lupum – Beware the wolf disguised as a lamb.

Publicly articulating details of the many attack scenarios I’ve thought of would be irresponsible. Instead, it’s time to start the conversation around the problem that our international community is currently faced with and how we can protect our precious open-source.

Again, this blog post is about getting the conversation started.

Over the last few weeks, I’ve met with high profile MVP’s and a few corporate executives that share similar quality and security concerns that I’m sharing in this blog post.

For the purpose of this blog post, “packages” refers to open-source JavaScript packages that are added to Nodejs, or JavaScript web applications using a package manager.

I’ll have a section down below for compiled downloads such as NuGet, Visual Studio Gallery, and the Visual Studio Marketplace.

Proposal Goals

  • Not add any burdens to the open-source developer
  • Provide package consumers a measured level of confidence in the package and its dependencies
  • Raise the quality of packages by having them evaluated
  • Have repositories provide evaluation services and reporting for their packages


Package evaluation is performed in the cloud.  An MVP friend also thought about a command line tool that could be used locally.

Package evaluation should be opt-in.  Allow developers wanting their packages evaluated to submit their package for evaluation. An opt-in approach would not add any burdens to developers not wanting to participate, while at the same time, driving up quality for the packages that are evaluated, giving those developers additional credibility for their efforts.

Consumers of packages could choose to use evaluated packages or continue to use non-evaluated packages at their own risk.

Evaluation and Download

Where packages are evaluated (centralized vs. non-centralized) is a topic that will generate much discussion and debate.

Where evaluated packages are downloaded from (centralized vs. non-centralized) is another topic that will generate much discussion and debate.

Evaluation Metrics

A standard set of metrics is applied to JavaScript packages, yielding a consistent evaluation report, enabling consumers to easily compare packages.

Below is a short “starter list” of metrics. Additional metrics should include the warnings such as those that npm emits when packages are installed.

Most evaluation metrics are yes or no.  Some are numeric; others are a simple list. When a package is evaluated, all of its dependencies are also evaluated. A package’s evaluation can only be as good as its weakest dependency.

  • Package signed
  • Included software license
  • Number of dependencies
  • Number of dependencies with less than ten lines of JavaScript
  • Package is out of date
  • Package has warnings
  • Have out of date dependencies
  • Has dependencies with warnings
  • Has unit tests
  • Has 100% unit test coverage
  • All tests pass
  • Makes network calls
  • Writes to file system
  • Threat assessment
  • Package capabilities (what API’s are being used)

NuGet, Visual Studio Gallery, Visual Studio Marketplace

NuGet, Visual Studio Gallery, and Visual Studio Marketplace serve compiled code which is evaluated differently than JavaScript. Microsoft will need to determine the best way to evaluate and report on these packages.


This proposal affects developers and infrastructures from all over the world.

As a software engineer, I know that while there will be challenges, the problems identified in this proposal are solvable.

Getting big corporations and government to proactively and cooperatively, take on and complete a task because it’s the right thing to do is a challenge that must be initiated.

Waiting until there is a problem and then trying to stem the tide and roll back the damage is a poor strategy.  Benjamin Franklin said, “an ounce of prevention is worth a pound of cure,” he is correct.

I honestly do not believe getting funding for a project of this scope will be any problem.

Next Steps

Big players need to meet and solve this problem.

Developers, start the conversation in your sphere of influence and contact big players and let them know your concerns.  Request that they take proactive action now.


Have a great day.

Just a grain of sand on the worlds beaches.



  1. ferventcoder

    “Just know that with every change, even when they are good for security and the community at large, does not mean that the entire community will like the changes.”

    We have similar concerns and thoughts with Chocolatey. Chocolatey is built on an enhanced NuGet model, so it is decentralized but has one “centralized” community source package repository.

    If you are open to meeting up, I’d be interested in talking the road ahead from the trenches of implementing and thinking about most of what has been presented here.

    The community repository, because it is the most used repository (and the default on install), is public and accepts community contributions, so it has the most need for moderation. We’ve had moderation in place for 2 years. Moderation is a double-edged sword – it slows down the process of getting things out for folks because it requires a human to say yes to every package after the other automated checks have passed, but it builds more trust in the approved packages.

    Just know that with every change, even when they are good for security and the community at large, does not mean that the entire community will like the changes. We’ve increased security recently by requiring download checksums in packages and have seen some consternation surrounding this. We were planning a longer transition to this requirement but due to a FossHub security incident we needed to move quicker to close the loop. We needed to start requiring it so that the binaries that are validated in the moderation process are the same binaries that the end user receives.

    When we introduced moderation 2 years ago, we had a lot of push back from package maintainers (and probably lost some of the community in the process), but we’ve added quite a bit of automation to this process to ensure that the human part of the process is very quick and just catching the things a computer can’t easily find. We run a validator that validates quality of package and looks for high level issues, a verifier that runs the package code in a controlled environment, and a scanner that submits binaries that a package has or downloads to VirusTotal to get a second opinion against 50-60 scanners (packages are usually represented by binaries). Even with the push back from maintainers, the greater community however told us they loved moderation – we saw the number of installs increase from 5 million over the first three years to 30 million over the next year (25 million installs in 1 year!).

    Package signing is something we are moving towards (and have for years). Other things to keep in mind – the security threat model continues to move, so whatever is put in place needs to be flexible enough to handle answering new threats as well (and hopefully easily).


    • Karl

      Hi Rob,

      Sent you an email, let’s get the ball rolling over a Skype.

      Agree with your remarks. Community move slow, which is why I recommended opt-in.




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s