launchd traps: WatchPaths ignores `touch`, and your keychain is locked over SSH
launchd is the only supported way to run background work on macOS, and most of what's written about it stops at "here's a plist." The failures below all come from running a real automation stack on launchd for months. Each one failed silently - the job didn't error, it just didn't do what we thought.
1. WatchPaths does not fire on a bare touch
The classic flag-file pattern: a deploy script touches /tmp/rebuild-requested, a LaunchAgent with WatchPaths on that file wakes up and rebuilds. It works - until the flag file already exists.
touch on an existing file updates only metadata (mtime). launchd's WatchPaths is backed by kernel file events that do not treat a metadata-only update as a change. Result: the deploy script "succeeded," the watcher slept for two hours, and the pipeline looked mysteriously dead.
The fix is one character longer than the bug:
# broken (existing file): metadata-only, watcher sleeps
touch /tmp/rebuild-requested
# works: content write, watcher fires instantly
date > /tmp/rebuild-requested
Audit every writer of every flag file. We found the broken form in three separate scripts, each written by someone who had seen the pattern work (on a not-yet-existing file) once.
2. Your SSH session is not your GUI session
macOS has a sharp, undocumented-feeling split between the GUI login session (the one with your unlocked keychain, your Accessibility grants, and a screen to show dialogs on) and everything else - SSH sessions, cron, and launchd daemons.
Consequences we hit, in increasing order of confusion:
codesignfails over SSH witherrSecInternalComponent. The login keychain is locked in an SSH context and the signing key's ACL wants a GUI prompt it can't show. Everything works when the same command runs from Terminal.app.- TCC prompts have no surface. A GUI-session process that triggers a permission check gets a dialog. A launchd agent that triggers the same check gets… nothing visible, and blocks or crashes. We watched a Bluetooth API call SIGABRT from a LaunchAgent when the identical code ran fine from Terminal - Terminal held the TCC grant; the agent had no way to even ask.
- A privileged parent launders permissions.
sshdon a box with remote administration typically holds Full Disk Access, and children inherit it - which is why "it works over SSH but not from launchd" is a real (and deeply confusing) failure signature: the SSH path was borrowing sshd's FDA the whole time.
The pattern that resolves most of this: put the privileged operation (signing, TCC-guarded file access) in a small watcher that runs in the GUI login session, and have your headless contexts communicate with it through flag files (with content writes - see trap #1).
3. launchd's environment is nearly empty
No PATH beyond the system default, none of your shell profile, no env vars your tools expect. Every script a LaunchAgent runs must either use absolute paths or explicitly source an environment file. The failure mode, again, is silence: the binary isn't found or an endpoint variable is empty, the job exits, and unless you gave it a StandardErrorPath, the evidence went nowhere.
While you're at it: /tmp is cleared on reboot. State that must survive a restart belongs under ~/, not /tmp - a rule we learned by watching a cache "mysteriously" reset.
4. The watcher can only build what it can see
One more silent no-op class, as a bonus. If launchd runs your build script via a symlink, then $(dirname "$0") inside the script resolves to the symlink's directory - not the real script's. Any sibling-relative path ($SCRIPT_DIR/ci/...) now points somewhere that may not exist, your [ -x ... ] guard quietly evaluates false, and the whole block skips forever with exit code 0.
Similarly: if the plist invokes /bin/zsh /path/script.sh, the explicit interpreter overrides the script's bash shebang. Bash-isms in that script don't error under zsh so much as behave differently - we lost a pre-compile step for hours to an array-expansion idiom that zsh silently no-ops.
Takeaways
- Flag files: write content, never bare-
touch. - Anything needing keychain or TCC belongs in the GUI login session; headless contexts signal it via files.
- "Works over SSH, fails from launchd" usually means an inherited grant (often sshd's FDA) was doing the work.
- Give every job
StandardErrorPath, absolute paths, and an explicit environment - then verify behavior, not exit codes. launchd's failure mode is almost always a successful-looking silence.
Related posts
Ad-hoc code signing silently orphans your macOS TCC permissions
The root cause is how TCC (Transparency, Consent, and Control) decides whether the thing asking for access is "the same program" you approved last time.
NSWindow released me twice: a crash from 1998 and the window-level ladder nobody documents
Two AppKit lessons from building a menu-bar assistant daemon that puts overlays, toasts, and cards on screen all day.
A 26,000-line Swift app with no Xcode: what bare `swiftc` taught us
Our menu-bar assistant daemon is a Swift/AppKit app that grew to ~26K lines - and it has never opened Xcode.