Auditing NPM Modules in Microservices

Where I work at Beamery, we follow Microservices Architecture and have a good 200+ git repositories. With time, these reports become bloated with old NPM modules that are either outdated or not used anymore at all. I wanted to find a way to quickly audit and clean this mess.

I came across the great npm-check that:

• Tells you what's out of date.
• Provides a link to the package's documentation to decide if you want the update.
• Kindly inform you if a dependency is not used in your code.
• Works on your globally installed packages via -g.
• Interactive Update for less typing and fewer typos via -u.
• Supports public and private @scoped/packages.
• Supports ES6-style import from syntax.
• Upgrades your modules using your installed version of npm, including the new npm@3, so dependencies go where you expect them.
• Works with any public npm registry, private registries, and alternate registries like Sinopia.
• Does not query registries for packages with private: true in their package.json.
• Emoji in a command-line app, because command-line apps can be fun too.
• Works with npm@2 and npm@3, as well as newer alternative installers like ied and pnpm.

To automate running npm-check across all of our reports and generate one coherent report, I created a simple bash-it plugin. The plugin, at its core, does the following:

# Generate an NPM report using the npm-check module to inspect the state of our npm modules
# The function will check if npm-check is installed and install it otherwise
# The report will be generated in the root directory and will be called npm-report.txt

generate_npm_report() {
    if command_exists npm-check ; then
        find . -maxdepth 1 -type d \( ! -name . \) -exec bash -c "cd '{}' && printf 'Examining NPM modules for '{}'' && echo '{}' >> ../npm-report.txt && npm-check >> ../npm-report.txt" \;
    else
        printf 'npm-check module was not found. Installing now:';
        npm install -g npm-check
        find . -maxdepth 1 -type d \( ! -name . \) -exec bash -c "cd '{}' && printf 'Examining NPM modules for '{}'' && echo '{}' >> ../npm-report.txt && npm-check >> ../npm-report.txt" \;
    fi
}

# Some helper functions to check if a certain command exists
command_exists () {
    type "$1" &> /dev/null ;
}

This script is executed at the root folder that contains all of your repositories, will execute the npm-check command, and aggregate the results in the npm-report.txt at the root directory where you executed your script. The script also checks first if the npm-check command exists or if we need to install it via an npm install -g npm-check.

To go step by step inside the main function:

• find . -maxdepth 1 -type d \( ! -name . \): Will find all the directories within one level down of the current folder
• printf 'Examining NPM modules for '{}'' will just print out a message indicating which folder we are currently examining
• echo '{}' >> ../npm-report.txt will print out the folder name examined in the output file npm-report.txt
• npm-check >> ../npm-report.txt This will execute the npm-check command and pipe out the result into the output file

Auditing NPM Modules

After knowing the various modules used, I cleaned my file, pasted the results in an Excel sheet, sorted the cells, and created a subTotal on the count. This generated a list of all my NPM modules and their respective count.

Cleaning out unused NPM modules

The previous function gives us an idea of the modules and frequency used. However, we might have a bunch of unused modules that were left over old code and will just increase the size of our containers with no actual use.

We can easily clean out those modules by taking advantage of the npm-clean and plug that in a similar wrapper as the function above:

# Clean unused NPM modules from each repository
# The function will check if npm-clean is installed and install it otherwise

clean_npm_modules() {

    if command_exists npm-clean ; then
        find . -maxdepth 1 -type d \( ! -name . \) -exec bash -c "cd '{}' && printf 'Cleaning NPM modules for '{}'' && echo '{}' >> ../npm-clean-report.txt && npm-clean >> ../npm-clean-report.txt" \;
    else
        printf 'npm-check module was not found. Installing now:';
        npm install -g npm-clean
        find . -maxdepth 1 -type d \( ! -name . \) -exec bash -c "cd '{}' && printf 'Cleaning NPM modules for '{}'' && echo '{}' >> ../npm-clean-report.txt && npm-clean >> ../npm-clean-report.txt" \;
    fi
}

# Some helper functions to check if a certain command exists
command_exists () {
    type "$1" &> /dev/null ;
}

I hope this helps you in cleaning out your repositories as well.