Your File Watcher Deleted the File You Just Saved

by

The setup is about as simple as file-based automation gets: watch a folder, copy anything written there into a second folder, and — because a stale mirror is worse than no mirror — remove from the target whatever gets removed from the source. Four lines of chokidar and you’re done.

Then someone hits Cmd-S in their editor, and the file vanishes from the target. Not corrupted, not half-copied. Gone. Sometimes it comes back a moment later; sometimes it doesn’t. The source file is fine the whole time.

TL;DR: Most editors don’t save into your file, they write a temp file and rename() it into place. A naive watcher sees unlink then add and, with delete-mirroring on, races itself into deleting the target. The fix is three ordered defenses: awaitWriteFinish so half-written files never get copied, a per-file debounce so a bundler’s event storm becomes one copy, and a delay on every delete that gets cancelled the moment the path reappears. All three are in copy-watch, which is where the code below comes from.

Why Does Saving a File Look Like a Delete?

Because for a lot of software, it is one. An atomic save means the editor never leaves your file in a partially-written state: it writes the new content to a temporary file next to the original, fsyncs it, and then rename()s the temp over the target. rename() on the same filesystem is atomic, so a reader either sees the entire old file or the entire new one — never a truncated middle.

Excellent property. Terrible thing to watch naively. The kernel events you get are, in order:

unlink   notes.md
add      notes.md

Or, depending on the editor, a supporting cast of files you never asked about:

  • vim writes a probe file literally named 4913 into the directory to test whether it can create files there, then deletes it.
  • Some macOS editors stage the whole thing in a hidden .sb-a1b2c3d4 directory next to the file.
  • Copying onto exFAT or SMB volumes leaves AppleDouble siblings (._notes.md) carrying the resource fork.
  • iCloud Drive leaves notes.md.icloud placeholders for files that are visible in Finder but not actually on disk.

So a watcher that maps events one-to-one onto filesystem operations has already made two mistakes before any of your code runs: it will faithfully mirror junk, and it will faithfully mirror a delete that was never a delete.

Why Not Just Copy on Every Event?

Because add doesn’t mean "this file is finished." It means "a directory entry appeared." A large asset written by a build step will fire add while it’s still a few kilobytes of a few megabytes, and you’ll copy the truncated version — with no error, because nothing failed. Truncated output that nobody reports is the expensive kind.

Chokidar’s answer is awaitWriteFinish: don’t emit until the file’s size has stopped changing for a while.

const watcher = chokidar.watch(srcRoot, {
  ignoreInitial: !initial,
  usePolling: poll,
  interval,
  binaryInterval: interval,
  // Verhindert, dass halb geschriebene Dateien kopiert werden.
  awaitWriteFinish: {
    stabilityThreshold: stability,   // default 300ms
    pollInterval: Math.min(100, interval),
  },
  // ...
});

300ms is a compromise, not a law. It’s the price of every copy — nothing is mirrored before it elapses — and it needs to be longer than the longest pause your writer takes mid-file. A bundler streaming output locally is comfortable at 300ms. A file arriving over a slow network mount is not; raise it there and accept the latency.

That handles one write. It does not handle the second problem: a bundler rewriting the same file four times in 200ms as it settles. Each of those is a legitimate, complete write, so awaitWriteFinish dutifully emits four events, and you do four copies where the first three are already garbage. Hence a second, separate timer — a per-file debounce, defaulting to 50ms:

const copyTimers = new Map();

const schedule = (map, key, delay, fn) => {
  clearTimeout(map.get(key));
  if (delay <= 0) {
    map.delete(key);
    fn();
    return;
  }
  map.set(key, setTimeout(() => { map.delete(key); fn(); }, delay));
};

Note the shape: keyed per path, and every new event for that path cancels the pending one. That’s the primitive the delete fix needs too.

How Do You Make a Delete Safe?

You don’t act on unlink when you receive it. You schedule the delete, and you let a subsequent add for the same path cancel it.

const deleteTimers = new Map();

const onWrite = (type) => (file) => {
  // Ein neu erscheinender Pfad annulliert eine noch offene Löschung.
  clearTimeout(deleteTimers.get(file));
  deleteTimers.delete(file);
  schedule(copyTimers, file, debounce, () =>
    run(type, file, (to) => copyFile(file, to, retries)),
  );
};

watcher
  .on('add', onWrite('copy'))
  .on('change', onWrite('update'))
  .on('unlink', (file) => {
    if (!deleteRemoved) return;
    clearTimeout(copyTimers.get(file));
    copyTimers.delete(file);
    schedule(deleteTimers, file, deleteDelay, () =>
      run('delete', file, (to) => rm(to, { force: true })),
    );
  });

Both directions cancel each other. An unlink drops any pending copy — no point copying a file that’s gone. An add drops any pending delete — the atomic save’s rename landed, so the "delete" was never real. The grace period (--delete-delay, default 400ms) is simply how long you’re willing to wait to find out which of the two it was.

Pick it deliberately: too short and a slow rename slips through as a real delete, which is the original bug wearing a smaller hat; too long and genuinely deleted files linger in the mirror. 400ms covers a local rename() by three orders of magnitude and is imperceptible to a human deleting a file on purpose. Over SMB, raise it.

Worth being explicit about what this is not: it is not a correctness guarantee, it’s a heuristic with a tunable window. There is no event that says "this was an atomic save." You are inferring intent from timing, and the honest thing to do is name the assumption in the flag’s default rather than bury it.

What Should the Ignore List Actually Contain?

The tempting move, once you’ve seen 4913 and ._notes.md and .sb-a1b2c3d4 go past, is to ignore every dotfile and be done with it. Don’t. That rule throws away .htaccess, .env, and the entire .well-known directory — for a folder being mirrored into a webroot, those are three of the files most likely to matter.

So the default list is deliberately narrow and enumerated:

export const DEFAULT_IGNORE = [
  /(^|[\\/])\.git([\\/]|$)/,
  /(^|[\\/])node_modules([\\/]|$)/,
  /(^|[\\/])\.DS_Store$/,          // macOS Finder-Metadaten
  /(^|[\\/])\.Spotlight-V100([\\/]|$)/,
  /(^|[\\/])\._[^\\/]+$/,          // AppleDouble Resource-Forks auf FAT/exFAT/SMB
  /\.icloud$/,                     // iCloud-Platzhalter
  /\.sb-[a-z0-9]+$/i,              // Atomic-Save-Verzeichnisse
  /~$/, /\.swp$/, /\.tmp$/, /\.crdownload$/, /\.part$/,
  /^4913$/,                        // vim-Testdatei beim Speichern
];

Every entry there is a name some piece of software chose for a file it intends to delete itself. That’s the actual membership rule, and it’s a much better filter than "starts with a dot."

Two more guards belong in the same category — cheap checks that turn a silent disaster into a startup error:

if (srcKey === destKey) {
  throw new Error('createCopyWatcher: "dest" darf nicht "src" sein');
}
if (destKey.startsWith(srcKey + path.sep)) {
  throw new Error(
    'createCopyWatcher: "dest" liegt innerhalb von "src" — das erzeugt eine Endlosschleife',
  );
}

A target nested inside the source is a feedback loop: every copy is a write inside the watched tree, which triggers another copy. It fills a disk in the time it takes to notice. And the comparison itself needs care on macOS, where the same path can arrive as NFD or NFC and comparisons are case-insensitive:

function normalizePath(p) {
  const normalized = path.resolve(p).normalize('NFC');
  return process.platform === 'darwin' || process.platform === 'win32'
    ? normalized.toLowerCase()
    : normalized;
}

Skip the normalize('NFC') and ~/Sites/Café compares unequal to itself depending on which program handed you the string.

What Doesn’t This Replace?

rsync. If all you want is for B to look like A, this has been solved for decades and solved better:

rsync -a --delete ./dist/ ~/Sites/preview/

Pair it with fswatch and you have the same behaviour in two lines of shell, with none of the timing heuristics above — because rsync compares state instead of reacting to events, which sidesteps the entire atomic-save problem rather than defending against it.

A watcher earns its place only when something has to happen per file: purge a cache entry, trigger a reload, kick a deploy, transform the file on the way through. That’s what the onEvent hook is for, and it’s the only reason to accept the extra machinery.

It also doesn’t preserve extended attributes, Finder tags, or resource forks — fs.cp copies content and mode, nothing more. Mirroring signed bundles or design files? Use ditto --rsrc --extattr. For build artifacts and source, it’s irrelevant.

What’s the Generalizable Lesson Here?

  • A filesystem event is a statement about a directory entry, not a statement about intent. unlink + add is one save operation about as often as it’s two separate ones, and nothing in the event stream tells you which.
  • When you can’t observe intent, encode the ambiguity as a tunable delay with a documented default — and say in the docs that it’s a heuristic. A hidden 400ms assumption is a bug report waiting to happen; a documented one is a flag.
  • Use separate timers for separate concerns. Bundling repeat writes (debounce) and waiting out a possible rename (delete-delay) look similar and want different durations; merging them means one of the two is always wrong.
  • Make destructive operations cancellable and let the cheap signal cancel the expensive one. A pending delete that any new write annuls is strictly safer than a delete that has to be undone.
  • "Ignore all dotfiles" is the wrong abstraction for temp files. The real category is "files some program created and intends to remove itself" — enumerate those, and keep .env and .well-known working.

The other half of getting this tool to run at all — the sudo prompt, the macOS privacy dialog, and the EMFILE crash, none of which is actually about admin rights — is covered in the file watcher that looked like a permissions problem.