Skip to main content
  1. Posts/

My commands vanished with exit 0. The culprit was a file named env.

··1441 words·7 mins·
Nick Liu
Author
Nick Liu
Building infrastructure for Facebook Feed Ranking at Meta. Previously at Walmart, Twitter, AWS, and eBay. MS in Computer Science at Georgia Tech.
Table of Contents
Hardening My Dotfiles - This article is part of a series.
Part 2: This Article
`env -u VAR command` did nothing. Exit code 0, no output, no error. A different command, same thing. A different variable, same thing. Any invocation that started with `env` just quietly evaporated. What finally made the problem visible was a `git init` that reported success while creating no `.git` directory at all.

That was a year ago. The case closed last month, and the culprit was not uv, not some third-party installer, not anything exotic. It was this repo’s own bootstrap script. The thing that caught it was the regression test I had written for the original incident.

This post is three silent failures from the same repo, plus that ending. They share a shape worth naming.

Case 1: the executable that should have been sourced
#

uv’s installer drops a file at ~/.local/bin/env. It is a PATH snippet, a few export lines meant to be sourced by your shell profile. It is not a program.

At some point, something gave it an executable bit. I could not figure out what at the time, so that part of the case stayed open. But the consequences were mechanical from there:

uv installer drops ~/.local/bin/env
(a PATH snippet, meant to be sourced)

something marks it executable
(culprit unknown, case left open)

~/.local/bin sits second in PATH,
ahead of /usr/bin

it now shadows /usr/bin/env

'env ... cmd' runs the snippet instead:
sets a few variables, exits 0,
never runs cmd

symptom: git init succeeds,
no .git exists

uv installer drops ~/.local/bin/env
(a PATH snippet, meant to be sourced)

something marks it executable
(culprit unknown, case left open)

~/.local/bin sits second in PATH,
ahead of /usr/bin

it now shadows /usr/bin/env

'env ... cmd' runs the snippet instead:
sets a few variables, exits 0,
never runs cmd

symptom: git init succeeds,
no .git exists

Every env -u X cmd, every shebang-adjacent env trick, every script that spelled out env for portability: all of them ran a five-line PATH snippet and stopped. With exit 0, because the snippet ran fine. It was just never asked to run your command, since sourcing semantics and executing semantics are different contracts.

The fix was one line:

chmod -x ~/.local/bin/env

Finding the line took the better part of a debugging session. So the incident became a permanent test, with the story written into the comment:

# uv's installer drops a PATH snippet at ~/.local/bin/env meant to be
# SOURCED; if it ever becomes executable it shadows /usr/bin/env (PATH
# puts ~/.local/bin second) and silently swallows every 'env ... cmd'
# invocation with exit 0. That exact trap cost a debugging session.
run_test "env resolves to /usr/bin/env (no executable shim)" \
    "[ \"\$(command -v env)\" = /usr/bin/env ]"

There is also a generalized version: a scan that flags any executable in ~/.local/bin shadowing a system command. The specific trap gets a named test; the class of trap gets a sweep.

Case 2: the fixture that reformatted my real repo
#

I was writing a fixture test for the secret scanner: create a throwaway git repo in a temp directory, plant a fake credential, confirm the scanner catches it. Standard stuff.

The test runs inside the yadm pre-commit hook. yadm exports GIT_DIR and GIT_WORK_TREE for its own purposes, and here is the part I did not respect enough: those environment variables outrank git -C <path>. The -C flag changes the working directory; the environment still decides which repository you are talking to.

So the fixture’s git init reinitialized the real yadm repository. It flipped core.bare and broke the repo’s configuration. Recovery turned out to be one setting, with zero history lost:

yadm gitconfig core.bare false

But the lesson went into the suite in capital letters:

# CRITICAL: every call strips GIT_DIR/GIT_WORK_TREE — the yadm
# pre_commit hook exports them, and an ambient GIT_DIR silently
# redirects fixture git commands at the REAL yadm repo (this once
# flipped its core.bare).
GL_GIT="env -u GIT_DIR -u GIT_WORK_TREE git"
$GL_GIT -C "$GL_TMP" init -q

The suite now also runs a hostile-environment simulation: it deliberately plants a decoy GIT_DIR and confirms the fixtures ignore it. And the fake credential is assembled at runtime, so no secret-shaped literal ever exists in the repo for a scanner to trip on:

printf 'aws_key = "%s%s"\n' "AKIA" "ZZZQK9X2M4P7L3TQ" \
    > "$GL_TMP/leak.txt"

Case 3: the file manager that shrugged
#

yazi 26 renamed a fetcher config field from id to group. My config still said id. When yazi hits a config it cannot parse, it does not error; it falls back to its presets and carries on looking almost normal. The git and mactag columns just stopped being enriched, and I only noticed weeks later, by accident.

The canary here is two words:

yazi --version    # exits 1 when the config fails to parse

That exit-code behavior is something I tested, not something I read somewhere. It now runs in the test suite and again after upgrades, as a schema check. When yazi next renames a field, the failure will be loud and same-day instead of silent and discovered by luck.

The ending: a year later, the case closed itself
#

Here is where case 1 gets its resolution, and it did not come from detective work.

An unrelated idempotency audit of the bootstrap script added one final step to it: after bootstrapping, run the full test suite as self-verification. The very first self-verification run came back red.

The bootstrap script contained a blanket chmod +x ~/.local/bin/*. The “scripts must be executable” convention, applied to a whole directory. It had just re-armed the uv trap, a year after I defused it, and it had presumably been the original armer all along. One line took down three tests at once: the env resolution test, the PATH shadow sweep, and the gitleaks fixtures, whose env -u calls were being swallowed again.

# before: the whole directory gets the executable bit.
# uv's source-me env snippet is in there, so the trap gets
# re-armed on every bootstrap run.
chmod +x ~/.local/bin/*

# after: only scripts yadm actually tracks get chmod'd.
# untracked files (including uv's env) never gain the bit.

The full lifecycle reads like this: symptom, defusal, regression test, then a year of quiet, then the test catches the original root cause in the act of reoffending, and the root cause gets fixed. The gap between step three and step four was twelve months. The gap between “bootstrap gained self-verification” and “cold case closed” was a few minutes.

The common shape
#

All three cases had no error message. All three returned success. All three were a system helpfully falling back to something instead of failing: a snippet standing in for a binary, an environment variable standing in for a flag, presets standing in for a config.

Silent fallback is the expensive kind of bug. A loud failure costs you five minutes. A quiet substitution costs you weeks, because nothing marks the moment where reality diverged from your model of it.

Institutionalizing the paranoia
#

Every incident in this post exists in the test suite as a named check. The suite grew from 74 checks to 105 across this overhaul, and almost all of that growth is “we got bitten here” items. Case 1 is the proof that the habit pays twice: the test written to protect the future is the same one that closed the past.

The other byproduct is a small document, docs/upgrade-watch.md, pairing every config file that tracks an upstream schema with a canary that fails loudly:

| Config file                 | Canary                  |
| --------------------------- | ----------------------- |
| ~/.config/yazi/yazi.toml    | yazi --version          |
| ~/.config/atuin/config.toml | atuin doctor            |
| ~/.tmux.conf                | throwaway-server parse  |
| ~/.local/bin/env (uv)       | suite `env` check       |

Lessons
#

  • Environment variables outrank the flags on your command line. git -C does not protect you from an ambient GIT_DIR.
  • A file that is meant to be sourced becomes a command-swallowing trap the moment it gains an executable bit and a good PATH position.
  • Tools that silently fall back to defaults on a bad config need a canary with a real exit code. Test the canary’s behavior; do not assume it.
  • Write the regression test even when the root cause is still unknown. Mine waited a year and then caught the culprit red-handed.
  • Bootstrap scripts should only touch files they own. A wildcard chmod is a loaded gun pointed at the future.

References
#

  • uv (the installer that ships the ~/.local/bin/env snippet)
  • yazi (the config-schema rename landed in yazi 26; exit-code canary behavior verified on this machine)
  • yadm hooks documentation (the pre-commit hook environment where GIT_DIR/GIT_WORK_TREE are exported)
  • The repo and its test suite: nickboy/dotfiles
Hardening My Dotfiles - This article is part of a series.
Part 2: This Article

Related

Five wrong answers in one day. One nearly deleted 10 GB of coursework.

··1903 words·9 mins
`mdls kMDItemLastUsedDate` returned `(null)` for Microsoft Word. I read the null as "never opened" and put Office on a removal list: 10.1 GB, four apps. One last check saved me. My home directory held 150 Office documents, a conference presentation edited two weeks earlier, and a PowerPoint lock file, which only exists while the file is open. The proof that the null was misleading had been sitting in my own diagnostic report for an hour. That was one of five. In a single day of hardening this machine, five different tools told me things that were not true. None of the answers looked like an error. Each one arrived as a clean, confident finding, and under each one a check had quietly failed or asked the wrong question. All five had the same shape underneath. Once I could name the shape, I stopped falling for it.

My dotfiles had a no-exceptions test gate. It had never run once.

··1331 words·7 mins
My dotfiles repo has a CLAUDE.md, and the CLAUDE.md has a rule in bold: every commit must pass the test suite, no exceptions. Within the first hour of an audit this July, I learned that this rule had been enforced exactly zero times since the day it was written. The hook file existed, its contents were correct, it even had its executable bit. It was just sitting at a path that yadm stopped reading a major version ago. No error message. No warning. To yadm, a hook in the wrong place and no hook at all are the same thing.

My diagrams rendered on refresh and vanished on click. The head never loaded.

··1010 words·5 mins
A reader clicking from my homepage to a post with a diagram got a block of raw mermaid source. The same reader pasting that post's URL directly got a rendered diagram. Same page, same build, same browser. The difference was the click, and the bug had been live on this site for months across every diagram, every math formula, and every chart, because I had only ever tested pages by loading them directly. 🧪 Tested with Blowfish 2.10x · htmx 2.0.10 Two features, both reasonable, one collision # This site has htmx’s hx-boost on the body: internal navigation swaps page content in place instead of doing full page loads, which keeps transitions smooth. Separately, the Blowfish theme is smart about heavy libraries: mermaid, KaTeX, and Chart.js bundles are only included on pages that use them, injected into the <head> of exactly those pages.