Introducing Node Sentinel File Watcher

Node Whatnow?

Node Sentinel File Watcher (NSFW). It’s a file watching module built for Node.js. I built NSFW to overcome an obstacle we were encountering while developing GitKraken, and I released it as an ongoing open-source project. It’s actually entirely safe for work. Ideal for it, even.

NSFW is a native Node module, which means it is developed and written to run natively on an operating system without an interpreter (JavaScript uses an interpreter, whereas a language like C++ compiles to machine code).

NSFW has a file watching implementation for Linux, MacOS, and Windows, which are wrapped and bound for the node runtime to utilize. Thanks to its native implementation, NSFW is able to watch recursive directories on all operating systems in a very performant way.

What Problem Does NSFW Solve?

At this time, Node has pretty poor support for file watching. Every operating system has a different capacity to watch directories. Here’s what some celebs have been saying on Twitter:

NSFW solves the poor file watching experience on Node by utilizing 3 low-level file watching utilities written in C++ and targeted for the Linux, MacOS, and Windows operating systems.

NSFW does most of its work on a separate thread, giving it big performance gains over the built-in Node file system (FS) watcher API. NSFW queues file events on a separate thread and batches file change dispatches into a single callback. That callback can be throttled internally to prevent spamming the JavaScript/C++ bridge.

This means NSFW doesn’t slow down JavaScript applications, even when they’re under a load of large FS operations.

Linux, MacOS, and Windows each ship with their own file watching APIs. Since NSFW’s watch utility is targeted specifically at each of those APIs, it means the experience of using the module is consistent across all three OSs. NSFW fills in the gaps for each API so they’re all consistently feature-complete:

• Linux: The Inotify file watching system does not perform recursive directory watching, so NSFW builds and maintains a recursive watch tree for you.
• MacOS: FSEvents is known to produce inconsistent file events (the file event bitmask becomes corrupted if events occur too quickly), so NSFW stats and disambiguates file change events for you.
• Windows: Supports all targeted needs out of the box. (RECORD SCRATCH) That’s right, Windows has the best native support. I said it.

Why This Was an Important Problem for the Axosoft Dev Team to Solve

GitKraken is currently the primary consumer of NSFW. A good Git client should not ship with a refresh button because it should automatically know when things change.

NSFW is essential to the smooth, cross-platform experience of GitKraken, as it helps the app respond quickly and accurately to changes in a repository it is viewing. NSFW is a quiet and humble, no-frills background process. It doesn’t make waves. It doesn’t talk through movies, chew loudly, or snore, but its transparency is its strength; if you notice it, it’s more than likely because something isn’t working. So cheers to watching files silently and effectively – something made easy by the NSFW module.

Development Hurdles

Oh yes.

After fiddling around with the project through the 0.x.x months (aka The Dark Times), I learned a lot about how each underlying file watcher API works, including their caveats, demands, and broken bits.

After spending a week putting together a complete system diagram, I scrapped the entire 0.x.x project and rebuilt the project to handle the differences of each operating system in a planned way. I also did away with the C++/CLI interface and opted for ReadDirectoryChangesW in Windows.

A Couple of Lessons Learned

1. Predicting a system’s architecture without first dirtying your hands with the core features, means you’ll probably end up throwing away your project, dirtying your hands with the core features, and starting the project over. As a new developer, be prepared to throw away your prototypes.
2. I’m not sure if C++/CLI and Node should ever be a thing. Ever.

Check out NSFW, and use it for your own projects.

Install NSFW