Skip to main content
  1. Posts/

My dotfiles had a no-exceptions test gate. It had never run once.

··1331 words·7 mins·
Nick Liu
Author
Nick Liu
Building infrastructure for Facebook Feed Ranking at Meta. Previously at Walmart, Twitter, AWS, and eBay. MS in Computer Science at Georgia Tech.
Table of Contents
Hardening My Dotfiles - This article is part of a series.
Part 1: This Article
My dotfiles repo has a CLAUDE.md, and the CLAUDE.md has a rule in bold: every commit must pass the test suite, no exceptions. Within the first hour of an audit this July, I learned that this rule had been enforced exactly zero times since the day it was written. The hook file existed, its contents were correct, it even had its executable bit. It was just sitting at a path that yadm stopped reading a major version ago.

No error message. No warning. To yadm, a hook in the wrong place and no hook at all are the same thing.

This was supposed to be a cleanup
#

The plan was modest: go through all 82 tracked files and delete some stale configuration. What actually happened was closer to an archaeology dig, and every layer had something dead in it that looked alive from the surface. Five findings, in the order they turned up.

Finding 1: the hook on the dead path
#

yadm 2.x read hooks from ~/.yadm/hooks/pre-commit, with a hyphen. yadm 3.x reads them from $YADM_DIR/hooks/pre_<command>, so a commit hook lives at ~/.config/yadm/hooks/pre_commit, with an underscore. My machine runs yadm 3.5. My hook was still parked at the 2.x path.

# yadm 2.x (dead, but the file was still there)
~/.yadm/hooks/pre-commit

# what yadm 3.x actually reads: $YADM_DIR/hooks/pre_<command>
~/.config/yadm/hooks/pre_commit

A directory move and one character, hyphen to underscore, and a rule labeled “no exceptions” died quietly for about a year. The fix took a minute. The part worth keeping is the comment that now sits at the top of the hook, so the path contract is written where the next person (me) will trip over it:

#!/bin/bash

# yadm pre-commit hook (yadm 3.x reads $YADM_DIR/hooks/pre_<command>,
# i.e. this file must live at ~/.config/yadm/hooks/pre_commit).
# Runs the full test suite (which includes markdownlint) before
# allowing a commit. Bypass in emergencies: yadm commit --no-verify

Finding 2: thirteen CI jobs, zero teeth
#

The repo’s CI had 13 jobs. Thirteen green circles is a comforting sight. Then I looked at which ones could actually stop a bad commit, and the answer was none that mattered: the critical steps had continue-on-error or a trailing || true scattered on them.

Thirteen jobs bought me a lot of reassurance and almost no protection. Whether a pipeline will actually refuse a bad commit is a separate property from how much of it there is, and nothing about the green circles tells you which one you have.

Finding 3: launchctl said yes and did nothing
#

The daily-maintenance launch agent had been silently stopped since July 3. The root cause is a genuinely nasty little contract: launchctl load exits 0 even when the agent is disabled in the override database, so the command reports success while the service stays down.

# exits 0 even when the agent is disabled in the override database
launchctl load ~/Library/LaunchAgents/com.daily-maintenance.plist

# what actually works: enable explicitly, then bootstrap
launchctl enable gui/$UID/com.daily-maintenance
bash ~/install-daily-maintenance.sh

“The command succeeded” and “the service is running” are separate claims. I now treat any launchd interaction that only checks exit codes as unverified.

Finding 4: the zombie updater
#

A duplicate brew-update automation had been running since February 2023. Two and a half years of doing the same work twice, and nobody noticed, because both copies succeeded. Redundant success is invisible in a way redundant failure never is.

Finding 5: the editor frozen in time, with a full paper trail
#

This one is the reason the post exists. Neovim had been frozen on the June 30 nightly build for five weeks before any symptom appeared. When it finally surfaced, it did so sideways: neo-tree called nvim_win_resize, an API added in a newer nightly than the one I was running, and blew up with “attempt to call a nil value”.

The causal chain took a while to untangle:

bob compiles a fresh Neovim
nightly every day

July: bob moves its data dir to
~/Library/Application Support/bob

a half-finished install there wedges
every 'bob install nightly'
('Couldn't find bob.json')

PATH still points at the abandoned
proxy under ~/.local/share

Neovim silently frozen
at the June 30 build

five weeks later: neo-tree calls an
API that build does not have

the failure was written to the
maintenance log every single day

and read by no one

bob compiles a fresh Neovim
nightly every day

July: bob moves its data dir to
~/Library/Application Support/bob

a half-finished install there wedges
every 'bob install nightly'
('Couldn't find bob.json')

PATH still points at the abandoned
proxy under ~/.local/share

Neovim silently frozen
at the June 30 build

five weeks later: neo-tree calls an
API that build does not have

the failure was written to the
maintenance log every single day

and read by no one

The wedged install I confirmed against bob’s own dev-branch source rather than guessing. And there is a general trap in here for anyone on nightlies:

" trap: every 0.13-dev nightly satisfies this condition,
" so it gates nothing, including the weeks before the API existed
if has("nvim-0.13")
  " ...
endif

has("nvim-0.13") is true for every 0.13-dev build, first to last. As a version gate for a feature that landed mid-cycle, it is no gate at all.

The immediate fix was three parts: unwedge the stuck install, point PATH back at the real bob, and add a boot canary to the test suite:

# if the editor cannot start, this goes red today, not in five weeks
# when some plugin happens to call an API that is not there
nvim --headless -c q

But the part that actually stings is this: the failure was in the maintenance log every day for a month, described accurately each time. It just landed in a file nobody reads. So the structural fix went to the output side. The maintenance summary now sends a desktop notification whenever any task fails, and the message layout assumes it will be truncated: log location first, at most three failures listed, the rest folded into a count.

# log location first (notifications get truncated, so the most
# important information leads), max 3 failures, rest folded
body="Failures logged in $LOG"
[ "$total" -gt 3 ] && body="$body (+$((total - 3)) more)"

What shipped
#

The cleanup turned into 22 single-concern commits, one problem per commit so that any of them can be reverted alone.

MetricBeforeAfterVerified how
CI jobs135coverage went up, not down: fed the gates a broken .sh, a broken .md, and a broken KDL config; all three rejected
Local test suitenone74 checksborn in this audit; at 105 today
zsh startup398 ms254 mshyperfine --warmup 3 'zsh -i -c exit'
Working pre-commit gatenoyescommits with failing tests are refused

Two of those rows deserve a sentence. The CI shrink is the point of finding 2: fewer jobs, strictly more protection, and the protection is proven by deliberately feeding the pipeline broken files rather than by looking at green circles. And the startup number comes from hyperfine because an unreproducible number is a mood, not a measurement.

Lessons
#

  • Automation you have never seen fail is indistinguishable from automation that does not exist. Their observable behavior is identical.
  • Validate a gate by deliberately breaking something it should catch. A gate that has only ever seen good inputs is unproven by construction.
  • Failures must arrive somewhere you actually look. A log line is not an alert; it is a diary entry.
  • Counting CI jobs measures reassurance. Counting what CI refuses measures protection.
  • When a tool renames its config path across a major version, every consumer of the old path fails silently. Write the path contract into the file itself.

References
#

  • yadm hooks documentation (the pre_<command> naming and $YADM_DIR lookup)
  • hyperfine (startup-time benchmarking)
  • bob (the Neovim version manager; wedge behavior confirmed against its dev-branch source)
  • The repo this all happened in: nickboy/dotfiles
  • launchctl behavior observed on macOS on this machine; see man launchctl for the enable/bootstrap subcommands
Hardening My Dotfiles - This article is part of a series.
Part 1: This Article

Related

Managing Dotfiles Like a Pro with Yadm

··878 words·5 mins
Every developer eventually reaches the point where their configs become too valuable to lose. Here's how I use **yadm** to manage my macOS dotfiles with automated testing, daily maintenance, and a pre-commit workflow that keeps everything in check. For me, the turning point was spending a weekend setting up a new MacBook and realizing I couldn’t reproduce my environment reliably. That’s when I started managing my dotfiles properly.

Five wrong answers in one day. One nearly deleted 10 GB of coursework.

··1903 words·9 mins
`mdls kMDItemLastUsedDate` returned `(null)` for Microsoft Word. I read the null as "never opened" and put Office on a removal list: 10.1 GB, four apps. One last check saved me. My home directory held 150 Office documents, a conference presentation edited two weeks earlier, and a PowerPoint lock file, which only exists while the file is open. The proof that the null was misleading had been sitting in my own diagnostic report for an hour. That was one of five. In a single day of hardening this machine, five different tools told me things that were not true. None of the answers looked like an error. Each one arrived as a clean, confident finding, and under each one a check had quietly failed or asked the wrong question. All five had the same shape underneath. Once I could name the shape, I stopped falling for it.

Three layers of secret defense for a public dotfiles repo. One was decorative.

··1190 words·6 mins
My dotfiles repo is public, which means any slip with a credential is permanent. History rewrites do not un-leak a key that a scraper already saw. So the defense cannot be one layer, and the interesting part of layering is not the count of tools. It is that each layer intercepts at a different moment: one before the commit exists, one at the moment of push, one sweeping the entire history in CI. The uncomfortable part, and the reason this post belongs to this series: one of my three layers used to be a decoration.