Every bulk replace exited zero. The damage was in the lines I did not delete

The numbers your replace tool returns say nothing about health

It is worth being precise about what we mean when we say a replace "succeeded".

The exit code of sed tells you whether the file could be read and written. The match count tells you the intended lines were hit. Neither says anything about the lines that were not hit. git diff is no better: deleted lines show up in red, but a surviving line that has lost its meaning has no colour of its own.

Every tool in this chain looks at the matched side. The breakage is almost always on the unmatched side.

As an indie developer running several apps at once, my replace targets naturally span multiple projects. A replace steps over the scale at which you can open each file and check it by eye. Being fast and being verifiable turned out to be separate properties.

This gets sharper when the target is AI-generated code. Generated output repeats structurally similar lines and names things regularly. A condition that would hit three places in hand-written code hits thirty in generated code, and every extra hit is another chance for a neighbouring line to be caught in the blast. If your workflow is generate, bulk-fix, submit, there is no step in it where that collateral damage would surface.

I wrote earlier about what a green build does and does not guarantee. This class of breakage sits below that line. A file can compile perfectly well and still render a raw translation key on screen because the dictionary lost an entry.

Four invariants a replace must preserve

So I stopped trying to read diffs and started asking a machine whether certain properties still hold. Four of them, in the order I rely on them:

1. Structural symmetry. If braces, brackets, parentheses, and code fences balanced before the replace, they must balance after. Strings and comments get flattened first so their contents are not counted.
2. Parseability. A file that parsed as JSON before must parse after. Stranded commas are caught here without ambiguity.
3. Invariant counts. Things the replace was never supposed to touch — export declarations, headings — must not decrease. A drop means something was caught in the blast.
4. Deleted-line adjacency. Check whether the line immediately above a deleted line ends in a form that expects continuation: ,, {, =>, and so on.

The fourth is a different kind of statement. One through three are evidence of definite breakage; the fourth is only a suspicion. Treating them identically makes the tool unusable, as the measurements below show plainly.

The guard script

No dependencies. Snapshot before the replace, check after it.

#!/usr/bin/env node
// replace-guard.mjs — find the lines that broke next to the ones you deleted
//   node replace-guard.mjs snapshot .rg-before src locales   before
//   node replace-guard.mjs check    .rg-before src locales   after
import { readFileSync, writeFileSync, mkdirSync, existsSync, readdirSync, statSync } from "node:fs";
import { join, relative, dirname } from "node:path";
 
const FENCE = "`".repeat(3);
const PAIRS = [["{", "}"], ["[", "]"], ["(", ")"]];
// Things a replace must never reduce. Extend per project.
const INVARIANTS = [/\bexport\s+(default|function|const)\b/g, /^\s*##\s+/gm];
const TEXT = /\.(tsx?|jsx?|mjs|json|mdx?|ya?ml)$/i;
 
function walk(dir, out = []) {
  for (const name of readdirSync(dir)) {
    if (name === "node_modules" || name.startsWith(".")) continue;
    const p = join(dir, name);
    statSync(p).isDirectory() ? walk(p, out) : TEXT.test(p) && out.push(p);
  }
  return out;
}
 
// Flatten strings, comments and fences before counting brackets
function strip(src) {
  return src
    .replace(new RegExp(FENCE + "[\\s\\S]*?" + FENCE, "g"), "")
    .replace(/\/\*[\s\S]*?\*\//g, "")
    .replace(/\/\/[^\n]*/g, "")
    .replace(/"(?:[^"\\\n]|\\.)*"/g, '""')
    .replace(/'(?:[^'\\\n]|\\.)*'/g, "''")
    .replace(/`(?:[^`\\]|\\.)*`/g, "``");
}
 
function shape(src) {
  const s = strip(src);
  const counts = {};
  for (const [open, close] of PAIRS) {
    counts[open] = (s.split(open).length - 1) - (s.split(close).length - 1);
  }
  counts[FENCE] = (src.match(new RegExp("^" + FENCE, "gm")) || []).length % 2;
  counts.__inv = INVARIANTS.map((re) => (src.match(re) || []).length);
  return counts;
}
 
// Does this line expect the next one to continue it?
const CONTINUES = /[,{[(+\-*/=&|?:]\s*$|=>\s*$/;
 
function parseOk(file, src) {
  if (/\.json$/i.test(file)) {
    try { JSON.parse(src); } catch (e) { return e.message.split("\n")[0]; }
  }
  return null;
}
 
const [mode, snapDir, ...roots] = process.argv.slice(2);
const files = roots.flatMap((r) => walk(r));
 
if (mode === "snapshot") {
  for (const f of files) {
    const dest = join(snapDir, f);
    mkdirSync(dirname(dest), { recursive: true });
    writeFileSync(dest, readFileSync(f));
  }
  console.log(`snapshot: ${files.length} files -> ${snapDir}`);
  process.exit(0);
}
 
let failed = 0;
let warned = 0;
for (const f of files) {
  const after = readFileSync(f, "utf8");
  const snap = join(snapDir, f);
  if (!existsSync(snap)) continue;
  const before = readFileSync(snap, "utf8");
  if (before === after) continue;
 
  const [sb, sa] = [shape(before), shape(after)];
  const block = [];  // definite breakage
  const warn = [];   // worth a look
 
  for (const k of ["{", "[", "(", FENCE]) {
    if (sb[k] === 0 && sa[k] !== 0) block.push(`${k} no longer balances (${sa[k]})`);
  }
  sb.__inv.forEach((n, i) => {
    if (sa.__inv[i] < n) block.push(`invariant #${i + 1} dropped ${n} -> ${sa.__inv[i]}`);
  });
  const pe = parseOk(f, after);
  if (pe && !parseOk(f, before)) block.push(`no longer parses: ${pe}`);
 
  const beforeLines = before.split("\n");
  const afterSet = new Set(after.split("\n"));
  beforeLines.forEach((line, i) => {
    if (afterSet.has(line)) return;
    const prev = (beforeLines[i - 1] || "").trimEnd();
    if (prev && afterSet.has(beforeLines[i - 1]) && CONTINUES.test(prev)) {
      warn.push(`deleting line ${i} may have left the line above dangling: "${prev.slice(-48)}"`);
    }
  });
 
  if (block.length) {
    failed++;
    console.log(`\n✗ BLOCK ${relative(process.cwd(), f)}`);
    for (const p of [...new Set(block)].slice(0, 4)) console.log(`   ${p}`);
  } else if (warn.length) {
    warned++;
    console.log(`\n△ WARN  ${relative(process.cwd(), f)}`);
    for (const p of [...new Set(warn)].slice(0, 2)) console.log(`   ${p}`);
  }
}
console.log(`\nBLOCK ${failed} / WARN ${warned}`);
process.exit(failed ? 1 : 0);

Run against the four files from earlier:

✗ BLOCK src/LegacyBanner.tsx
   { no longer balances (-1)
   invariant #1 dropped 1 -> 0

△ WARN  src/PaywallScreen.tsx
   deleting line 3 may have left the line above dangling: "llScreen({ onClose }: { onClose: () => void }) {"

△ WARN  src/useCredits.ts
   deleting line 5 may have left the line above dangling: "  const spend = (n: number) =>"

✗ BLOCK locales/ja.json
   no longer parses: Expected double-quoted property name in JSON at position 58 (line 4 column 1)

BLOCK 2 / WARN 2

The same operation that returned 0 four times now stops with exit code 1 on two files of definite breakage.

Do not route suspicions through the same exit as evidence

This was the design decision I went back and forth on.

Two WARN lines appear above, and neither of those files is actually broken. In PaywallScreen.tsx a comment directly after a function's opening brace was removed. In useCredits.ts a comment directly after an arrow function's => was removed. Both remain perfectly valid syntax.

On this sample the adjacency heuristic scored zero out of two. On its own it is not a useful signal.

I kept it anyway, and this is why. The site in useCredits.ts looked like this:

  const spend = (n: number) =>
    // legacy: clamp until the new quota API ships
    setLeft((v) => Math.max(0, v - n));

It survived this time. Had the setLeft line also contained a matching token, the entire arrow function body would have gone and the file would not compile. WARN is not saying "this broke". It is saying "this shape can break". That is a three-second judgement call for a human, not grounds for failing a build.

Worth adding: this script fell into the same class of trap once. To count code fences it carried three backticks written literally into its source. The moment I pasted the script into an article code block, that literal paired with the surrounding fence and swallowed the following paragraphs as code. A routine that counts the contents of text broke because of how it was itself written down. The fix was extracting a const FENCE. Tools that process text forget, at their peril, that they are text.

Definite breakage and structural fragility are different things. Send them down the same pipe and the warnings become noise nobody reads. BLOCK drives the exit code; WARN just prints.

Three steps before touching generated code

For projects generated by Rork or a comparable builder, I keep this order fixed.

1. Snapshot immediately before the replace. git stash or a commit would work, but copying the target set outright is more reliable — it still works when the working tree is messy right after generation.

node replace-guard.mjs snapshot .rg-before app components locales

2. Run the replace. Unchanged from whatever you do today.

3. Gate the commit on the check.

node replace-guard.mjs check .rg-before app components locales && git add -A

The && is the whole point. One BLOCK and you never reach git add. That single line removes the path where the person who ran the replace moves on to the next task without noticing. Most defects that surface only in production arrive down a path exactly like that one — open, unwatched, and short.

Extend the invariant list per project. On an Expo project I add the number of plugins entries in app.config.ts and the key count of each file under locales/. Translation key counts should match across languages, so if one side drops, something was caught in the blast.

The change in expo prebuild defaults that regenerates the native directories belongs to the same family of accident: hand-made changes disappear, and nothing tells you at the moment they disappear. The habit of snapshotting carries over directly.

Add one checkpoint downstream of the replace

If you want to try one thing today, stop invoking the replace command directly and wrap those three steps in a shell function or an npm script. That is enough.

Missing 36 broken files out of 975 for several weeks was not a lapse of attention. There was a way to confirm the replace had succeeded, and no step anywhere in the process that confirmed the output was still healthy. The tools only look at the matched side. Someone has to be assigned to the other side, and that someone has to be added deliberately.

Some of those articles are still waiting to be fixed. At least they have stopped multiplying.