The Fastest Directory Crawler for NodeJS

⚡ Extremely Fast: Nothing beats fdir in speed. It can easily crawl a directory containing 1 million files in < 1 second.

💡 Stupidly Easy: fdir only has 2 functions; sync and async for crawling the file system synchronously or asynchronously.

🤖 Zero Dependencies: fdir uses pure NodeJS fs & path namespaces. Nothing else is ever touched.

🕺 Astonishingly Small: < 1KB in size

🔥 All Node Versions Supported: fdir runs everywhere on all Node versions (within reason). And it is unsurprisingly fastest there too.

🚄 Quickstart

You can install using npm:

$ npm i --save fdir

or Yarn:

$ yarn add fdir

It makes no difference to me.

const fdir = require("fdir");

// get all files in a directory synchronously
const files = fdir.sync("path/to/dir");

// or asynchronously
fdir.async("path/to/dir").then(/*blah blah blah*/);

And that's it.

📊 Benchmarks:

$ yarn benchmark

Specs:

• Intel i7 7th Generation (7700HQ)
• 16 GB of RAM
• 256 GB SSD
• OS: Manjaro Linux
• Directory Size: 7386 files

Notes:

• Some people asked that I benchmark no-op (without options) version of fdir. I did and found no performance difference. The results were identical. (I didn't include it here as it wasn't anything special.)

Node v13.11.0:

Synchronous (7386 files)	Asynchronous (7386 files)

Node v8.3.0:

Note: As latest version of rrdir doesn't support Node < 8, I had to use version 2.0.0. Everything else is fully updated.

Synchronous (7386 files)	Asynchronous (7386 files)

🚒 API:

fdir is very small so there's not much to the API.

fdir.sync(string, Options): String[]

This is often the fastest way to get files. However, it will block the main "thread" so use it with caution with large directories.

fdir.async(string, Options): Promise<String[]>

Not always the fastest but works without blocking the street, so that's a plus.

Options

Ah, the options. Not many of them. At least not as many as I'd hoped for.

includeDirs: boolean

Whether to include directories in the array returned.

default: false

excludeBasePath: boolean

Whether to exclude the base path for each file.

default: false

searchFn: Function

Use this to filter out files.

Example:

fdir.sync("node_modules", {
  searchFn: path => path.includes(".git")
});

default: undefined

maxDepth: number

The max number of levels fdir should crawl before stopping. The lower the faster.

default: undefined (i.e. infinity)

isExcludedDir: Function

Use this to exclude particular directories from being crawled.

Example:

const isExcludedDir = path => path.includes(".bin");
fdir.sync("node_modules", { isExcludedDir });

default: undefined

ignoreErrors: boolean

Ignore errors while traversing the directory.

default: false

And that's it.

⁉️ FAQs:

1. I looked at the code and there's nothing special. How is it so damn fast then?

Well, that's the whole point. fdir exists to prove to the "young" generation that you don't need to use special constructs or special methods to gain speed. Just a bit of patience and brains.

2. I found X library. I ran its benchmarks. It is faster than fdir!

Um. Well thank you for embarassing me (just joking). Do tell me the name of this library though. I will try to optimize fdir and reclaim the first spot 😄

3. You are doing X and Y wrong! Do Z and it will improve performance!

Yes. And I should probably do A, B & C too. The point is, did you run benchmarks with these suggestions? If you did and saw significant improvements, thank you. Now go open a PR :laugh:

4. Why create this? What's the point?

I know you don't care. Fine. There's no point behind this. It's "just for fun". No, wait. Actually, I created this, first of all, for me. I needed fast directory access in another app of mine, so fdir came into being.

5. Why are all the other libraries so slow?

Because they did not spend enough time optimizing it. Most developers give readability and cool code more importance than actual performance and usability. I have seen a library claiming to be the fastest by inverting the benchmarks. Literally. Gave me quite the scare until I went and fixed the benchmark. It was actually one of the slowest. :O

6. How long did it take you to create this?

Ummm. Maybe 18 hours? Make it a day.

7. Are you looking for a job?

Am I? Well, are you offering a job? If yes, I am interested. :D

8. Why should I care?

You shouldn't. But here's my email in case you do: thecodrr[at]protonmail.com. Don't worry, I don't bite.

ℹ️ Support

Would love if you buy me a cup of coffee right over here. Or just be, you know, polite and give me a star? Maybe even follow me?

🦮 LICENSE

Copyright (c) 2020 Abdullah Atta under MIT. Read full text here.