My agent's shell said 0 errors. The log had 48,299.

The command was ordinary:

log show --last 30m --predicate 'messageType == error OR messageType == fault' | wc -l

In my terminal it works. In the agent’s shell, log is not the macOS log tool. It is a zsh builtin that has nothing to do with logs.

What log meant in that shell

#

zsh ships a module called zsh/watch, and that module adds exactly one builtin command named log. Its job is to list the users currently logged in. It takes no arguments, so when it receives show --last 30m ... it complains and exits with status 1:

$ zsh -c 'whence -w log'
log: builtin
$ zsh -c 'log show --last 1m'
zsh:log:1: too many arguments

That error went to stderr, and the audit never looked at stderr. What it looked at was the pipeline, and a pipeline in zsh reports the status of its last command. The last command was wc -l, which happily counted zero lines of input and exited 0:

$ zsh -c 'log show --last 1m 2>/dev/null | wc -l; echo "status=$?"'
       0
status=0
$ zsh -c 'setopt pipefail; log show --last 1m 2>/dev/null | wc -l; echo "status=$?"'
       0
status=1

The zsh manual describes this default plainly: without PIPE_FAIL, the exit status of a pipeline is the status of its rightmost element. With pipefail set, the same command returns 1. The 0 on stdout stays the same either way, which is worth noticing: even with the option on, a script that reads only the printed number still sees a clean log.

“It works when I type it”

#

This is the part that cost the most, because I could not reproduce it.

macOS knows about the collision. The system /etc/zshrc contains this line:

$ grep -n 'disable log' /etc/zshrc
13:disable log

That disables the builtin so log falls through to /usr/bin/log. The catch is in the file name. zsh reads /etc/zshrc for interactive shells only. My terminal is interactive, so I never see the builtin. The shell my coding agent runs commands in is not interactive, so /etc/zshrc never runs, and the builtin wins.

I checked this from inside the agent’s own shell while writing this post:

$ echo "interactive? $([[ -o interactive ]] && echo yes || echo no)"
interactive? no
$ whence -w log ps ls
log: builtin
ps: alias
ls: alias

The two shells share my aliases and functions, because the agent’s shell loads a snapshot of them. They do not share the system file that cleans up the log name, and that one missing line is the whole difference between a log reader and a list of logged-in users.

Right exit code, wrong answer

#

Once I knew to look, log was not alone. Two of my aliases shadow real commands in both shells, and they fail in different ways:

The ps aux row is the one I am most wary of. procs treats any extra word as a keyword to match against user and command names, so aux becomes a filter instead of a set of flags. On the night of the original audit, it returned 4 rows where /bin/ps aux returned 943. Today, from the agent’s shell, it is 12 rows against 935. The exit code is 0 both times.

A loud failure like ps -Ao costs me a minute. A quiet one with the right exit code goes straight into a report as a fact.

The same bug, inside the suite meant to catch it

#

After writing the rule down, I went looking for the same shape in my own test suite, and it was already there:

# passed for every possible Brewfile, valid or not
brew bundle check --file="$BREWFILE" | head -1

My test runner judges a check by its exit status, and this pipeline’s status belongs to head -1, which succeeds as long as it can print a line. So the check passed no matter what the Brewfile contained. It now uses brew bundle list, which fails on a syntax error and stays quiet about packages that are simply not installed yet. That second property matters, because otherwise CI would go red for roughly 280 packages it never installs.

The rule I wrote about all this was also wrong, twice, before review caught it. My first draft said the log builtin exits 0. It exits 1, which is exactly why pipefail catches it. The draft also implied the builtin shadows log at my prompt, which it does not. I had written a rule about trusting unverified output, and I had not run the commands in the rule.

What I do now

#

• I spell it /usr/bin/log, every time, in scripts and in anything an agent will run.
• Scripts that count lines set pipefail, and they also check that the command produced output at all before they trust a count of zero.
• Before trusting output that looks empty, I check what the name resolves to in the shell that ran it, with whence -w or which -a, not in my own terminal.
• My agent instructions now list these shadowed names explicitly, with the row counts above as the example, so the next audit starts from the full path instead of rediscovering this.

Lessons

#

• A pipeline reports its last command’s exit status. Anything earlier in the chain can fail silently behind a wc -l or a head -1.
• Your agent’s shell is not your terminal. It skips the interactive startup files, so names your terminal has fixed can still mean something else there.
• “It works when I type it” is evidence about your shell only. Reproduce in the shell that actually ran the command.
• When a check can only fail quietly, give it a second check on its output. An exit code of 0 says the last command was happy, nothing more.
• A count of zero is a claim. Before writing it down, show that the query produced output under a broader filter.

References

#

• zsh manual: the zsh/watch module (the log builtin, which lists logged-in users)
• zsh manual: options, PIPE_FAIL (a pipeline’s status is its rightmost element’s by default)
• procs (non-numeric arguments are matched as keywords against user and command)
• log(1) on macOS for the real /usr/bin/log
• The rule and the fixed checks: nickboy/dotfiles