The Fastest Directory Crawler & Globber for NodeJS

⚡ The Fastest: Nothing similar (in the NodeJS world) beats fdir in speed. It can easily crawl a directory containing 1 million files in < 1 second.

💡 Stupidly Easy: fdir uses expressive Builder pattern to build the crawler increasing code readability.

🤖 Zero Dependencies*: fdir only uses NodeJS fs & path modules.

🕺 Astonishingly Small: < 2KB in size gzipped & minified.

🔥 All Node Versions Supported: Unlike other similar libraries that have dropped support for Node versions < 10, fdir supports all versions >= 6.

🖮 Hackable: Extending fdir is extremely simple now that the new Builder API is here. Feel free to experiment around.

* picomatch must be installed manually by the user to support globbing.

Support

Do you like this project? Support me by donating, creating an issue, becoming a stargazer, or opening a pull request. Thanks.

🚄 Quickstart

Installation

You can install using npm:

$ npm i fdir

or Yarn:

$ yarn add fdir

Usage

const { fdir } = require("fdir");

// create the builder
const api = new fdir().withFullPaths().crawl("path/to/dir");

// get all files in a directory synchronously
const files = api.sync();

// or asynchronously
api.withPromise().then((files) => {
  // do something with the result here.
});

Documentation:

I have written in-depth documentation here.

📊 Benchmarks:

Specs:

• CPU: Intel i7 7th Generation (7700HQ)
• RAM: 16 GB
• Storage: 256 GB SSD
• OS: Manjaro Linux
• Directory Size: < 2k files

Notes:

• Some people asked that I benchmark no-op (without options) version of fdir.
  • I did and found no performance difference. The results were identical. (I didn't include it here as it wasn't anything special.)
• Some other people were doubtful about the authenticity of these results due to frequency scaling, process overload, disk warmup, etc.
  • So I have updated the benchmark with new results that should resolve all those doubts. Here's the process I followed:
  • Hard shutdown the laptop (a couple of times just to be sure) to clear disk, ram cache etc.
  • Login directly to a TTY (avoiding any unnecessary process from starting).
  • Disable CPU Scaling using

    $ sudo cpupower frequency-set --governor performance

  • Run the benchmark

The Fastest Globber

Last updated: May 13, 2020 (fdir v3.3.0)

$ yarn bench:glob

glob pattern used: **.js & **/**.js

Node v13.13.0

Synchronous	Asynchronous

The Fastest Directory Crawler

Last updated: May 10, 2020 (fdir v3.0.0)

$ yarn bench

Node v14.2.0:

Synchronous	Asynchronous

Node v8.7.0:

Older versions of fdir (1.x & 2.x) used synchronous lstat call (lstatSync) in the asynchronous API to acheive speed on Node < 10. This has been fixed in fdir 3.0.0.

Synchronous

🦮 LICENSE

Copyright © 2020 Abdullah Atta under MIT. Read full text here.