Implementing hot reload for a ExpressJS and EJS site

When working on another personal web project I faced the issue of hot reloading project files on change. The web project is doing a bunch of server-side rendering using ExpressJS and EJS and I couldn’t find a nice hot reload system for it. There were some examples on using webpack for this purpose, but I don’t want to introduce a whole bundling system just for the hot reloading.

So I’m writing my own :).

Let’s outline what is the expected functionality to solve our problem:

1. Detect a changed file on the server,
2. Signal the browser from the server to perform a reload (ideally on only on the changed resource),
3. The browser receives the signal and reloads the resource.

All three listed points are rather simple problems to solve on their own, so the hardest part of this hot reload system is just gluing it all together ;).

Detecting changes

Since all of my machines, and the servers that this service will be deployed on, are running Linux, then there’s only one good solution that I know of — inotify.

From man inotify:

The inotify API provides a mechanism for monitoring filesystem
events.  Inotify can be used to monitor individual files, or to
monitor directories.  When a directory is monitored, inotify will
return events for the directory itself, and for files inside the
directory.

Someone had already written V8 bindings for inotify, but it appears that these bindings are for an older version of NodeJS and fail when compiling with my NodeJS 16.3. I found another file system watching library with a dang hard name to type (which I had to once again look up, even though I just used it :@) — chokidar. Chokidar appears to work fine, so I’ll proceed with using that.

After I discovered that the node-inotify project doesn’t compile I found out that NodeJS has it’s own watching functionality in the form of fs.watch(), but it has a huge problem (for me) — it doesn’t support recursive watches on Linux systems. If it would support them or if I only needed to watch files, then I would’ve used that.

Signalling the browser

To signal the browser we’ll need to get a message from the service to the back-end to the browser, for which there are two options: long polling or WebSockets. As the final solution has to only run during development, then browser support isn’t an issue. I prefer the WebSockets method, as it’s a bit cleaner and I don’t have to reconfigure express to use it.

So we’ll use WebSockets to emit a “file changed” signal, which I’ll wrap into a JSON object just in case I want to emit other types of signals in the future.

Let’s use the nice WebSockets server for NodeJS for the actual server portion. Taking the examples on their GitHub page and wrapping it together results in hmr.js:

const WebSocket = require('ws');
const chokidar = require('chokidar');

const wss = new WebSocket.Server({ port: 8080 });

const onFileChange = (path, stat) => {
    console.log(`Change file: ${path}`);

    const data = { type: 'change', path };
    console.log(wss.clients);
    wss.clients.forEach((client) => {
        if (client.readyState === WebSocket.OPEN) {
            client.send(JSON.stringify(data));
        }
    });
}

module.exports.addWatchDir = (dir) => {
    console.log(`New watch dir: ${dir}`);
    chokidar.watch(dir)
        .on('add', onFileChange)
        .on('change', onFileChange)
        .on('unlink', onFileChange);
};

Which is included from the main program as follows:

const hmr = require('./hmr');
hmr.addWatchDir('./pages/**/*');
hmr.addWatchDir('./static/**/*');

Receiving the signal from the browser

We’ve got our WebSocket server which emits “file changed” signals, now all that’s left is to perform a reload on the browser side.

The code must parse the received signal and then, depending on the type (of which there is only one, “change”), perform an action.

The resulting code is as follows:

function hmr_connect() {
    const ws = new WebSocket('ws://localhost:8080');
    ws.addEventListener('open', function (event) {
        console.log('HMR socket open');
    });

    ws.addEventListener('close', function (event) {
        console.log('HMR socket closed');
        setTimeout(hmr_connect, 1000);
    });

    ws.addEventListener('message', function (event) {
        const msg = JSON.parse(event.data);
        switch (msg.type) {
            case 'change':
                console.log('File changed: ' + msg.path);
                // NOTE: delay loading a bit in case it coincides with server
                // restart, which would result in browser not being able to
                // load the page.
                setTimeout(function() { location.reload() }, 500);
                break;
            default:
                console.error('Unsupported command verb: ' + msg.type);
                break;
        }
    });
}

// NOTE: delay first load because in my case the first attempt tended to fail
setTimeout(hmr_connect, 200);

To actually load the script then I included a script tag to load the hot reload script from my static folder.

Parting thoughts

Overall the solution seems to work fine in my testing. You should have at least an overview on how to implement a hot reload system ;).

The hot reload code should be put behind some development flags, so the JS won’t be included in production HTML and the server not created.

It’d be real neat to check which ExpressJS route the file belongs to (in case of template files) and then emit the changed file’s URL as well, so the browser side script could ignore changes to files that it isn’t displaying. I don’t consider implementing that to be worth the research effort right now, so it’s up to you, if you feel like sating your curiosity or find the described functionality useful.